Vehicle trajectory prediction near or at traffic signal

ABSTRACT

A system and method for determining a predicted trajectory of a human-driven host vehicle as the human-driven host vehicle approaches a traffic signal. The method includes: obtaining a host vehicle-traffic light distance d x  and a longitudinal host vehicle speed v x  that are each taken when the human-driven host vehicle approaches the traffic signal; obtaining a traffic light signal phase P t  and an traffic light signal timing T t ; obtaining a time of day TOD; providing the host vehicle-traffic light distance d x , the longitudinal host vehicle speed v x , the traffic light signal phase P t , the traffic light signal timing T t , and the time of day TOD as input into an artificial intelligence (AI) vehicle trajectory prediction application, wherein the AI vehicle trajectory prediction application implements an AI vehicle trajectory prediction model; and determining the predicted trajectory of the human-driven host vehicle using the AI vehicle trajectory prediction application.

STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH

This invention was made with government support under DE-EE0007212awarded by the U.S. Department of Energy. The government has certainrights in the invention.

TECHNICAL FIELD

This invention relates to methods and systems for determining apredicted trajectory of a human-driven vehicle as the vehicle approachesand/or departs from a traffic signal.

BACKGROUND

Autonomous driving has been more successful on highway roads than onurban city roads mainly due to the simplicity of the highway's drivingenvironment, including the absence of traffic signals and pedestrians.Realizing fully autonomous vehicles in urban driving environments ismore challenging due to the opposite reasons, such as the existence oftraffic signals, frequent interactions with human-driven vehicles, andthe presence of pedestrians. One of the major differences between urbancity driving and highway driving is the presence of traffic lights. Inurban driving, especially in the vicinity of traffic lights exemplifiedby signalized corridors or intersections, the motions of vehicles aremainly governed by traffic signals. People drive in such a manner so asto obey the traffic signals and properly respond to implicit rulesimposed by traffic lights. Examples of the implicit traffic rulesinclude stopping before a traffic light that is in a RED phase, andmaintaining a proper speed when approaching a traffic light that is in aGREEN phase and when there is no traffic causing the vehicle to slowdown (i.e., a free-flow situation).

SUMMARY

It has been discovered that it is important to predict how human driversrespond to traffic signals for purposes of successfully carrying outand/or improving autonomous driving in areas having traffic signals(e.g., traffic lights). If trajectories of surrounding human-drivenvehicles can be predicted, then these predictions can be leveraged andused in decision-making, trajectory planning, and control synthesis ofan autonomous vehicle (AV), as well as for various other uses. In lightof this, there is provided herein a system and method of predicting atrajectory of a human-driven vehicle as it approaches or is near atraffic signal, where the prediction is determined through use of anartificial intelligence (AI) vehicle trajectory prediction model. The AIvehicle trajectory prediction model is generated or composed in light oftwo observations: (1) behaviors of human-driven vehicles near trafficlights are mainly governed by traffic signals; and (2) the phase andtiming of traffic signals can be shared (or otherwise made available)through vehicle-to-infrastructure (V2I) communications ahead of time.Thus, the AI vehicle trajectory prediction model maps states ofhuman-driven vehicles and a corresponding state of a traffic signal(e.g., signal time and phasing (SPaT) information of the traffic signal)to trajectories of the vehicles, which each can be represented by alongitudinal acceleration of the vehicle.

In accordance with an aspect of the invention, there is provided amethod for determining a predicted trajectory of a human-driven hostvehicle as the human-driven host vehicle approaches a traffic signal,wherein the method is carried out by one or more electronic controllers.The method includes: obtaining a host vehicle-traffic light distanced_(x) and a longitudinal host vehicle speed v_(x) that are each takenwhen the human-driven host vehicle approaches the traffic signal;obtaining a traffic light signal phase P_(t) and a traffic light signaltiming T_(t), wherein the traffic light signal phase P_(t) represents aphase of the traffic signal taken when the human-driven host vehicleapproaches the traffic signal, and wherein the traffic light signaltiming T_(t) represents an amount of time elapsed since a last phasechange of the traffic signal taken when the human-driven host vehicleapproaches the traffic signal; obtaining a time of day TOD; providingthe host vehicle-traffic light distance d_(x), the longitudinal hostvehicle speed v_(x), the traffic light signal phase P_(t), the trafficlight signal timing T_(t), and the time of day TOD as input into anartificial intelligence (AI) vehicle trajectory prediction application,wherein the AI vehicle trajectory prediction application implements anAI vehicle trajectory prediction model; and determining the predictedtrajectory of the human-driven host vehicle using the AI vehicletrajectory prediction application.

In various embodiments, the method may include any of the followingfeatures or any technically-feasible combination of two or more of thesefeatures:

-   -   the method further includes obtaining a front vehicle state        X^(FV), wherein the front vehicle state includes a front-host        vehicle distance r_(t) and a front-host vehicle speed {dot over        (r)}_(t), and wherein the providing step further includes        providing the front vehicle state X^(FV) as input into the AI        vehicle trajectory prediction application;    -   the front vehicle state X^(FV) is obtained at the one or more        electronic controllers based on front vehicle base information        that is obtained at the front vehicle and then sent via        vehicle-to-vehicle (V2V) communications to the one or more        electronic controllers;    -   the traffic light signal phase P_(t) and the traffic light        signal timing T_(t) are obtained from a traffic light control        system that is present at an intersection where the traffic        light is located;    -   the predicted trajectory is obtained at an autonomous vehicle        that is approaching the traffic light and that is separate from        the human-driven host vehicle;    -   the method is carried out at the autonomous vehicle as the        human-driven host vehicle approaches the traffic light;    -   the autonomous vehicle obtains the traffic light signal phase        P_(t) and the traffic light signal timing T_(t) from a traffic        signal system located at an intersection where the traffic light        is located;    -   the autonomous vehicle receives the traffic light signal phase        P_(t) and the traffic light signal timing T_(t) via        vehicle-to-infrastructure (V2I) communications from roadside        equipment that is a part of the traffic signal system;    -   the autonomous vehicle receives the traffic light signal phase        P_(t) and the traffic light signal timing T_(t) from a traffic        signaling control system that is located remotely from the        traffic light;    -   the host vehicle-traffic light distance d_(x), the longitudinal        host vehicle speed v_(x) are obtained at the autonomous vehicle        via V2V communications with the host vehicle;    -   the vehicle-traffic light distance d_(x), the longitudinal host        vehicle speed v_(x), the traffic light signal phase P_(t), and        the traffic light signal timing T_(t) are each associated with        an associated time that is no more than a predetermined amount        different than another one of the associated times;    -   the AI vehicle trajectory prediction model is or includes a        neural network;    -   the AI vehicle trajectory prediction model is a deterministic or        a model that predicts one or more most-probable trajectories;    -   the AI vehicle trajectory prediction model is a probabilistic        model that returns a probability distribution of predicted        trajectories, and wherein the predicted trajectory is obtained        by sampling a trajectory from the probability distribution of        predicted trajectories;    -   the neural network is a mixture density network;    -   the neural network is a deep neural network; and/or    -   the method further includes the step of causing an autonomous        vehicle to obtain the predicted trajectory of the human-driven        vehicle, wherein the autonomous vehicle is configured to: obtain        the predicted trajectory of the human-driven vehicle, and carry        out an autonomous vehicle operation based on the predicted        trajectory of the human-driven vehicle.

In accordance with another aspect of the invention, there is provided amethod for determining a predicted trajectory of a human-driven hostvehicle as the human-driven host vehicle approaches a traffic signal,wherein the method is carried out by one or more electronic controllers.The method includes: obtaining a host vehicle-traffic light distanced_(x) and a longitudinal host vehicle speed v_(x) that are each takenwhen the human-driven host vehicle approaches the traffic signal,wherein the host vehicle-traffic light distance d_(x) and thelongitudinal host vehicle speed v_(x) each have an associated time;obtaining a front vehicle state X^(FV), wherein the front vehicle stateincludes a front-host vehicle distance r_(t) and a front-host vehiclespeed {dot over (r)}_(t); receiving one or more wireless signals thatindicate a traffic light signal phase P_(t) and an traffic light signaltiming T_(t), wherein the traffic light signal phase P_(t) represents aphase of the traffic signal taken when the human-driven host vehicleapproaches the traffic signal, and wherein the traffic light signaltiming T_(t) represents an amount of time elapsed since a last phasechange of the traffic signal taken when the human-driven host vehicleapproaches the traffic signal, wherein the traffic light signal phaseP_(t) and the traffic light signal timing T_(t) each have an associatedtime, wherein the associated times of the host vehicle-traffic lightdistance d_(x), the longitudinal host vehicle speed v_(x), the trafficlight signal phase P_(t), the traffic light signal timing T_(t), thefront vehicle state includes a front-host vehicle distance r_(t), andthe front-host vehicle speed {dot over (r)}_(t) are within a maximumallowable time difference with respect to one another; obtaining a timeof day TOD; providing the host vehicle-traffic light distance d_(x), thelongitudinal host vehicle speed v_(x), the traffic light signal phaseP_(t), the traffic light signal timing T_(t), the front vehicle stateX^(FV), and the time of day TOD as input into an artificial intelligence(AI) vehicle trajectory prediction application, wherein the AI vehicletrajectory prediction application implements an AI vehicle trajectoryprediction model, and wherein the AI vehicle trajectory prediction modelis or includes a neural network; and determining the predictedtrajectory of the human-driven host vehicle using the AI vehicletrajectory prediction application.

In various embodiments, the method of the preceding paragraph mayinclude any of the following features or any technically-feasiblecombination of two or more of these features:

-   -   the host vehicle-traffic light distance d_(x) and the        longitudinal host vehicle speed v_(x) are both obtained at an        autonomous vehicle through receiving one or more wireless        signals from the human-driven host vehicle via        vehicle-to-vehicle (V2V) communications; and/or    -   the host vehicle-traffic light distance d_(x) and the        longitudinal host vehicle speed v_(x) are both obtained at the        autonomous vehicle through receiving one or more wireless        signals from a remote server.

BRIEF DESCRIPTION OF THE DRAWINGS

Preferred exemplary embodiments will hereinafter be described inconjunction with the appended drawings, wherein like designations denotelike elements, and wherein:

FIGS. 1A-1B depict a communications system that includes acomputer-based system and operating environment that can be used forcarrying out a method for obtaining a predicted trajectory of ahuman-driven vehicle as the human-driven vehicle approaches and/ordeparts from a traffic signal;

FIGS. 2A-2B depicts an exemplary scenario (a red-green (RG) scenario) inwhich the method discussed herein can be used to determine a predictedtrajectory of the human-driven vehicle as the human-driven vehicleapproaches a traffic signal;

FIG. 3 depicts an exemplary scenario in which a human-driven hostvehicle (HV) approaches a traffic signal with a front vehicle (FV) is infront of the host vehicle;

FIG. 4 depicts timing diagrams of three exemplary scenarios in which themethod discussed herein can be used to obtain a predicted trajectory ofthe human-driven vehicle as the human-driven vehicle approaches and/ordeparts from a traffic signal;

FIG. 5A illustrates an embodiment of an artificial intelligence (AI)vehicle trajectory prediction model according to a deterministic policy;

FIG. 5B illustrates an embodiment of an artificial intelligence (AI)vehicle trajectory prediction model according to a probabilistic policy;

FIG. 6 depicts a flowchart in which an AI vehicle trajectory predictionmodel that uses a deterministic policy or a probabilistic policy isobtained from off-line learning and that is then used to obtain atrajectory prediction of a human-driven vehicle as the human-drivenvehicle approaches and/or departs from a traffic signal;

FIG. 7 depicts a flowchart of an embodiment of a method for determininga predicted trajectory of a human-driven vehicle as the human-drivenvehicle approaches and/or departs from a traffic signal;

FIGS. 8A-8D depict predicted and actual trajectories of thedeterministic model for a G scenario according to two samples;

FIGS. 9A-9D depict predicted and actual trajectories of thedeterministic model for a Y scenario according to two samples;

FIGS. 10A-10D depict predicted and actual trajectories of thedeterministic model for a R scenario according to two samples;

FIGS. 11A-11D depict predicted and actual trajectories of thedeterministic model for a GY scenario according to two samples;

FIGS. 12A-12D depict predicted and actual trajectories of thedeterministic model for a YR scenario according to two samples;

FIGS. 13A-13D depict predicted and actual trajectories of thedeterministic model for a RG scenario according to two samples;

FIGS. 14A-14B depict predicted and actual trajectories of thedeterministic model for a GYR scenario according to one sample;

FIGS. 15A-15B depict predicted and actual trajectories of various modelsfor a RG scenario according to one sample;

FIGS. 16A-16B depict predicted and actual trajectories of various modelsfor a YR scenario according to one sample;

FIGS. 17A-17B depict predicted and actual trajectories of various modelsfor a GYR scenario according to one sample;

FIGS. 18A-18B depict predicted and actual trajectories of various modelsfor another GYR scenario according to one sample;

FIG. 19 depicts a graph illustrating calculation of three performancemetrics used to measure performance of the AI vehicle trajectoryprediction model or application;

FIGS. 20-25 represent results from an ablation study that is used tocompare the prediction errors of various models used for six scenarios(G, R, GY, YR, RG, GYR);

FIG. 26 depicts a graph illustrating the evaluation results on positionerrors that resulted from using the various models used to predict thetrajectory according to six (6) scenarios;

FIG. 27 depicts a graph illustrating the evaluation results on speederrors that resulted from using the various models used to predict thetrajectory according to six (6) scenarios; and

FIGS. 28-29 depict a sample trip observed that represents the yellowlight dilemma scenario in which the traffic signal changes from yellowto red.

DETAILED DESCRIPTION

The system and method described herein enables a trajectory of ahuman-driven vehicle to be predicted as the vehicle is approaching atraffic signal. As used herein, a human-driven vehicle is a vehiclewhose propulsion and/or steering is at least partially manuallycontrolled by a human driver. Thus, at least according to someembodiments, the method and/or system can utilize information fromvehicle-to-infrastructure (V2I) communications (including informationfrom roadside equipment (RSE), connected vehicles (CVs) (including thehuman-driven vehicle in some embodiments), and/or other devices/systems)to model the driving behavior of human drivers near traffic signals,which can then be used to predict a trajectory of a human-driven hostvehicle that is approaching (or departing from) or near a trafficsignal. At least according to some embodiments, the predicted trajectorycan be used by an autonomous vehicle (AV) that is not the same vehicleas (or separate from) the human-driven host vehicle. The predictedtrajectory can be represented by a predicted vehicle acceleration a,which can include both a predicted magnitude and a predicted directionof the acceleration of the human-driven host vehicle. The term “hostvehicle” as used herein refers to the vehicle whose trajectory is beingpredicted. In some embodiments, the acceleration that represents thepredicted trajectory can be a longitudinal acceleration a_(x), which isdefined as the acceleration of the vehicle along a direction that isparallel to the road on which the vehicle is traveling as it approachesthe traffic signal—this direction is referred to herein as thelongitudinal direction. Other trajectory information can be derived fromthis predicted acceleration (such as when it is used in conjunction withother information). This other trajectory information can include avehicle speed or velocity (e.g., a longitudinal speed or longitudinalvelocity) as the host vehicle travels approaches and/or travels throughthe traffic signal. In at least some embodiment, the predictedtrajectory is predicted by using a human policy model that maps a stateof the human-driven host-vehicle vehicle (X^(HV)) and a state of atraffic signal (X^(TL)) to an acceleration (a^(HV)) (or other trajectoryinformation) of the host vehicle. In one embodiment, the predictedtrajectory (X) is predicted by using a human policy model (ƒ_(d) for thedeterministic policy, ƒ_(p) for the probabilistic policy) that maps thehost-vehicle vehicle state (X^(HV)), the traffic signal state (X^(TL)),and a state of a front vehicle (X^(FV)) to an acceleration (a^(HV)) (orother trajectory information) of the host vehicle. In at least oneembodiment, the front vehicle state (X^(FV)) is not taken intoconsideration as a part of the trajectory prediction, and the humanpolicy model in such embodiments is denoted ƒ_(d) ^(NoFV) for thedeterministic policy and ƒ_(p) ^(NoFV) for the probabilistic policy (andsimply f^(NoFV) in general, which may refer to both). The human policymodels that perform this mapping can be composed and/or carried outusing an artificial intelligence (AI) technique or mechanism (e.g.,neural networks) and, thus, is referred to as an AI vehicle trajectoryprediction model. As used herein, “host vehicle” refers to the vehiclefor which the trajectory prediction is being carried out and “frontvehicle” refers to a vehicle that is in front of the host vehicle withrespect to the direction of travel along the road.

The method(s) and system(s) described herein implement (or can be usedto implement) one or more of the human policy models, which are alsoreferred to herein as AI vehicle trajectory prediction models (or“trajectory prediction model” for short) and which are each used todetermine a predicted vehicle trajectory of a human-driven host vehiclebased at least on a host vehicle-traffic light distance (d_(x)), alongitudinal host vehicle speed (v_(x)), a traffic light signal phase(P_(t)), an amount of time relative to the last (or next) phase changeof the traffic light (or “traffic light signal timing”) (T_(t)), and thetime of day (TOD). The host vehicle-traffic light distance (d_(x)) andthe longitudinal host vehicle speed (v_(x)) are both a part of the hostvehicle state (X^(HV)), and can be represented as X_(t) ^(HV):=[d_(t),v_(t)] for time t. It should be appreciated that, in some embodiments, asequence of host vehicle states, traffic light states, and/or frontvehicle states corresponding to times t-τ to t can be taken intoconsideration, where τ defines the size of the range that is taken intoconsideration, which can include any suitable number of states for agiven time within the range t-τ to t.

The AI vehicle trajectory prediction model is developed based ontraining data of a plurality of training events, where the training dataof each training event at least includes a host vehicle-traffic lightdistance d_(x), longitudinal host vehicle speed v_(x), a traffic lightsignal phase P_(t), an traffic light signal timing T_(t), and a time ofday TOD. That is, in one embodiment, for a given one of the plurality oftraining events, the training data includes or is representative of thedistance between the host vehicle and the traffic light (or “hostvehicle-traffic light distance d_(x)”), a longitudinal speed of the hostvehicle (or “longitudinal host vehicle speed v_(x)”), a phase of thetraffic light (or “traffic light signal phase P_(t)”), the elapsed timesince the last phase change of the traffic light (or “traffic lightsignal timing T_(t)”), and the TOD, where each of the hostvehicle-traffic light distance d_(x), the longitudinal host vehiclespeed v_(x), the traffic light signal phase P_(t), and the traffic lightsignal timing T_(t) are taken at a time corresponding to one another(i.e., at the same time or close in time (i.e., the time difference isless than a predetermined amount, which can be, for example, 0.1seconds, although this amount can be set for the particular applicationand/or environment in which the system/method is being used)) and thatcan be described by the TOD. In some embodiments, the training data canfurther include a front vehicle state (X^(FV)), which can include afront-host vehicle distance r_(t) and a front-host vehicle speed {dotover (r)}_(t). The training data can be used by a model learningapplication, which is an application that is specifically configured toimplement one or more artificial intelligence techniques so as tocompose the trajectory prediction model based on the training data.

Once the trajectory prediction model is learned, the model can beimplemented by a particular vehicle (e.g., the host vehicle, anothernearby vehicle), by a traffic signal system, by a remote facility (ormultiple remote facilities), or by a combination thereof. In oneembodiment, the trajectory prediction model is implemented by an AIvehicle trajectory prediction application that is stored on memory at anautonomous vehicle (AV) and executable at the AV using a processor.According to at least some embodiments, once the trajectory predictionmodel is composed via offline learning using the training data, then thetrajectory prediction model can be used to map current vehicleparameters to a predicted vehicle trajectory. In one embodiment, thetrajectory prediction model is implemented in an on-board AI vehicletrajectory prediction application (or “on-board trajectory predictionapplication” for short) and, in another embodiment, the trajectoryprediction model is implemented in an off-board AI vehicle trajectoryprediction application (or “off-board trajectory prediction application”for short). In yet another embodiment, the trajectory prediction modelis implemented in a combination of an on-board trajectory predictionapplication and an off-board trajectory prediction application. As usedherein, the “on-board trajectory prediction application” is anembodiment of the AI vehicle trajectory prediction application that isexecuted by a device that is a part of the vehicle or that travels withthe vehicle, such as a vehicle system module (VSM) or a personalwireless device (e.g., smartphone). And, as used herein, the “off-boardtrajectory prediction application” is an embodiment of the AI vehicletrajectory prediction application that is executed by a device that isnot a part of the vehicle and that does not travel with the vehicle,such as a remote facility or roadside equipment (RSE).

In some embodiments, the trajectory prediction model can be initiallylearned or composed through a learning process that uses the trainingdata and, once the trajectory prediction model meets certain performanceand/or accuracy measures, then the trajectory prediction model can beembodied in a AI vehicle trajectory prediction application as a computerprogram product that can be disseminated to various devices that canexecute the trajectory prediction application, such as one or moreremote servers of one or more remote facilities, one or more autonomousvehicle (AVs), RSEs, etc. Any initial learning of the trajectoryprediction model can thus be done at one or more remote facilities andthis initial learning can use training data that is obtained from avariety of sources, including third party sources and/orpreviously-recorded data of a vehicle OEM or municipal system (e.g., atraffic signal system operated or controlled by a municipality or othergoverning body). Once the trajectory prediction model is initiallytrained or developed, the trajectory prediction model can then bedeployed and used commercially. Further, according to at least someembodiments, after the trajectory prediction model is implemented foruse with a plurality of vehicles, the trajectory prediction model cancontinue learning through additional training data that is gathered bythe plurality of vehicles. The trajectory prediction model can thuscontinuously be improved.

According to at least some embodiments, the trajectory prediction modelis based on a neural network. In one embodiment, the trajectoryprediction model is based on a deterministic neural network (typicallydenoted with a “d” in the subscript, such as ƒ_(d)) and, in anotherembodiment, the trajectory prediction model is based on a probabilisticneural network (typically denoted with a “p” in the subscript, such asƒ_(p)). In other embodiments, the trajectory prediction model can bebased on a combination of a deterministic neural network and aprobabilistic neural network. In some embodiments, the probabilisticmodel can be used in scenarios where competing policies exist or can beused to obtain probabilistic contexts for the predicted trajectories.The trajectory prediction model is used to provide a trajectoryprediction of a host vehicle based on certain parameters pertaining tothe vehicle, which, according to many embodiments, includes at least thehost vehicle-traffic light distance d_(x), the host longitudinal hostvehicle speed v_(x), the traffic light signal phase P_(t), the trafficlight signal timing T_(t), and the time of day (TOD). As mentionedabove, the predicted trajectory can be represented by an acceleration,including the magnitude and direction of the acceleration. According tosome embodiments, the predicted acceleration can be used to derive othervalues, such as the velocity of the vehicle. And, in some embodiments,the predicted trajectory can be represented by a velocity, which canthen be used to derive an acceleration. Thus, according to variousembodiments, the trajectory prediction can include a predicted velocityand/or acceleration of the host vehicle as a function of time. In someembodiments, the trajectory prediction can be accompanied by anuncertainty (or confidence) score that represents a level of uncertainty(or confidence) in the trajectory prediction.

With reference now to FIGS. 1A and 1 , there is shown an operatingenvironment that comprises a communications system 1 and that can beused to implement the method disclosed herein. Communications system 1generally includes vehicles 10,11,12, one or more wireless carriersystems 13, a land communications network 14, a remote processingfacility 16, a municipal facility 18, a traffic signal system 20including a traffic signaling device 22, a constellation of GNSSsatellites 50, and a mobile device 60. The host vehicle 10 is ahuman-driven vehicle and is also a connected vehicle (CV). As usedherein, a connected vehicle (CV) is a vehicle that is capable ofcarrying out data communications with a remote server, such as thoselocated at the remote facility 16. The front vehicle 11 is a connectedvehicle (CV). However, in other embodiments, the host vehicle 10 and/orthe front vehicle 11 is not a connected vehicle. In some embodiments,the front vehicle 11 is an autonomous vehicle. The autonomous vehicle(AV) 12 is a connected vehicle and an autonomous vehicle, which, as usedherein, an autonomous vehicle (AV) is a vehicle that is categorized as aLevel 2 or higher according to the National Highway Traffic SafetyAdministration (NHTSA) or Society of Automotive Engineers (SAE) standardJ 3016-2018. In some embodiments, the vehicle 10 can be a Level 3 (orhigher), a Level 4 (or higher), or a Level 5 (or higher).Vehicle-to-infrastructure (V2I) communications include communicationsthat can be carried out between vehicles and roadside equipment (RSE)(e.g., RSE 26), as well as communications between vehicles and remotenetworks, such as remote facility 16 and/or municipal facility 18. Suchcommunication system may be one example that can carry outvehicle-to-infrastructure (V2I) communications. It should be understoodthat the disclosed method can be used with any number of differentsystems and is not specifically limited to the operating environmentshown here. Also, the architecture, construction, setup, and operationof the system 1 and its individual components are generally known in theart. Thus, the following paragraphs simply provide a brief overview ofone such communications system 1; however, other systems not shown herecould employ the disclosed method as well.

Wireless carrier system 13 may be any suitable cellular telephonesystem. Carrier system 13 is shown as including a cellular tower 15;however, the carrier system 13 may include additional cellular towers aswell as one or more of the following components (e.g., depending on thecellular technology): base transceiver stations, mobile switchingcenters, base station controllers, evolved nodes (e.g., eNodeBs),mobility management entities (MMEs), serving and PGN gateways, etc., aswell as any other networking components required to connect wirelesscarrier system 13 with the land network 14 or to connect the wirelesscarrier system with user equipment (UEs, e.g., which include telematicsequipment in vehicles 10,11,12), all of which is indicated generally at31. Carrier system 13 can implement any suitable communicationstechnology, including for example GSM/GPRS technology, CDMA or CDMA2000technology, LTE technology, etc. In general, wireless carrier systems13, their components, the arrangement of their components, theinteraction between the components, etc. is generally known in the art.

Apart from using wireless carrier system 13, a different wirelesscarrier system in the form of satellite communication can be used toprovide uni-directional or bi-directional communication with thevehicle. This can be done using one or more communication satellites(not shown) and an uplink transmitting station (not shown).Uni-directional communication can be, for example, satellite radioservices, wherein programming content (e.g., news, music) is received bythe uplink transmitting station, packaged for upload, and then sent tothe satellite, which broadcasts the programming to subscribers.Bi-directional communication can be, for example, satellite telephonyservices using the one or more communication satellites to relaytelephone communications between the vehicles 10,11,12 and the uplinktransmitting station. If used, this satellite telephony can be utilizedeither in addition to or in lieu of wireless carrier system 13.

Land network 14 may be a conventional land-based telecommunicationsnetwork that is connected to one or more landline telephones andconnects wireless carrier system 13 to remote facility 16. For example,land network 14 may include a public switched telephone network (PSTN)such as that used to provide hardwired telephony, packet-switched datacommunications, and the Internet infrastructure. One or more segments ofland network 14 could be implemented through the use of a standard wirednetwork, a fiber or other optical network, a cable network, power lines,other wireless networks such as wireless local area networks (WLANs), ornetworks providing broadband wireless access (BWA), or any combinationthereof.

Remote facility 16 may be designed to provide the vehicle electronics ofany CVs, mobile device 60, and/or other wirelessly connected devices(WCDs) (i.e., a device that is capable of communicating data with aremote network, such as a remote server at the remote facility 16) witha number of different system back-end functions. The remote facility 16represents one or more remote facilities that house servers, memory, orother computing equipment that is used to carry out variousfunctionality described herein, including an AI learning process thatuses training data to generate or build the AI vehicle trajectoryprediction model. Additionally, the remote facility 16 can be used tocarry out or support an AI vehicle trajectory prediction application,which is an application that implements the AI vehicle trajectoryprediction model. Although a single remote facility is referred toherein, it should be appreciated that one or more remote facilities canbe used to implement the various backend (or off-board) functionalitydiscussed herein.

The remote facility 16 may include various components and may include awired or wireless local area network. The remote facility 16 includesone or more remote servers 17. The one or more remote servers 17 eachare an electronic controller and each include a processor and memory.The memory can store computer instructions that, when executed, carryout various functionality, such as one or more of the method stepsdiscussed herein, at least according to some embodiments. The remotefacility 16 can also include other computers or computing devices, aswell as other memory, databases, etc. Generally, the remote facility 16may receive and transmit data via a modem connected to land network 14.A database at the remote facility 16 can store vehicle information,trajectory data (e.g., training data, predicted trajectory data, and/oractual trajectory data), GNSS data, and any other data pertaining toWCDs. Data transmissions may also be conducted by wireless systems, suchas IEEE 802.11x, GPRS, and the like. In one embodiment, the remotefacility 16 may be used to implement at least part of one or moremethods disclosed herein. In some embodiments, the remote facility 16may receive GNSS data, trajectory prediction input data (i.e., data thatis, or is used to derive, the host vehicle-traffic light distance d_(x),the longitudinal host vehicle speed v_(x), the traffic light signalphase P_(t), the traffic light signal timing T_(t), front-host vehicledistance r_(t), a front-host vehicle speed {dot over (r)}_(t), and/orthe time of day TOD), and/or other data from vehicles 10,11, mobiledevices 60, and/or other WCDs. The remote facility 16 may then processthis data to predict vehicle trajectories (e.g., vehicle acceleration,vehicle velocity) and/or use this data as training data for generating,composing, building, or improving the AI vehicle trajectory predictionmodel and/or application.

Municipal facility 18 is a remote facility that is owned or controlledby a municipality or other governmental body. The municipal facility 18includes traffic signaling control system 19, which may include variouscomputers, databases, servers, and other computing devices. Trafficsignaling control system 19 may be used for controlling trafficsignaling devices, such as traffic signal 22, or may be used forproviding traffic signal state (X^(TL)), including the traffic lightsignal phase P_(t) and traffic light signal timing T_(t), to one or moreother devices or systems, such as the remote facility 16. The trafficsignal state can be stored in memory at the traffic signal system 20, ormay be stored at the municipal facility 18. It should be appreciatedthat, although the present example describes the traffic signalingcontrol system 19 being implemented at the municipal facility 18, inother embodiments, this system 19 can be implemented at anotherlocation, such as a server or database that is not owned or controlledby a municipality.

The traffic signaling control system 19 generates traffic control datathat can be sent to traffic signals or other traffic signaling devicessuch as pedestrian cross-walk lights and lane direction and closuresignals. The traffic signaling control system 19 may receive trafficsignal state data from one or more traffic sensors, such as inductanceloop detectors and/or video detectors, or from other roadside equipment(RSE) 26 that may be located at or near intersections. In someembodiments, the municipal facility 18 or the traffic signaling controlsystem 19 may receive information from vehicles 10,11,12, mobile devices60, and/or other WCDs via RSE 26, cellular carrier system 13, and/orland network 14. In one embodiment, the municipal facility 18 canreceive a data request from remote facility 16, such as a request fortraffic signal state information (e.g., signal phase and timing (SPaT)data, which can include, for example, traffic light signal phase P_(t)and traffic light signal timing T_(t)). In response to this request, themunicipal facility 18 can provide the requested data and/or other data,such as the traffic signal state information. The term “traffic signalstate information” refers to any information that represents the trafficsignal state X^(TL) or any part thereof.

Either or both of remote facility 16 and municipal facility 18 caninclude a computer-based system having one or more servers or computersthat include an electronic processor and memory. The processors can beany type of device capable of processing electronic instructionsincluding microprocessors, microcontrollers, host processors,controllers, vehicle communication processors, and application specificintegrated circuits (ASICs). The processors may execute various types ofdigitally-stored instructions, such as software or firmware programsstored in the memory, which enable the facility to provide a widevariety of services. For instance, the processors at the remote facility16 may be configured to execute programs or process data to carry out atleast a part of the method discussed herein. In one embodiment, theprocess can execute an application (e.g., computer program) that causesthe processor to perform one or more of the method steps usinginformation or data that is received from the one or more vehicles10,11,12, municipal facility 18, other remote facilities and/or servers,mobile devices 60, and/or other WCDs. The memory at remote facility 16and/or municipal facility 18 can include powered temporary memory and/orany suitable non-transitory, computer-readable medium such as differenttypes of RAM (random-access memory, including various types of dynamicRAM (DRAM) and static RAM (SRAM)), ROM (read-only memory), solid-statedrives (SSDs) (including other solid-state storage such as solid statehybrid drives (SSHDs)), hard disk drives (HDDs), magnetic or opticaldisc drives, or other suitable memory.

Traffic signal system 20 is depicted as including traffic signal 22,controller 24, and roadside equipment (RSE) 26, but can include anetwork switch, additional traffic signals or other types of trafficsignaling devices, a router, a modem, other network interface controlleror module, and/or a memory device. Any one or more of these componentscan be housed in the traffic signal and/or in a separate housing locatednear the traffic signal. In one embodiment, the controller 24 caninclude a processor and memory, and can be configured to operate thetraffic signal, for example, by activating and deactivating signalingcues (e.g., any visual, audible, or other indicator or notification thatcan be perceived by an operator on a roadway near or at the signalingdevice). The memory device at traffic signal can be a memory managementunit (MMU) and/or a non-volatile memory device, which can includepowered temporary memory and/or any suitable non-transitory,computer-readable medium such as different types of RAM (random-accessmemory, including various types of dynamic RAM (DRAM) and static RAM(SRAM)), ROM (read-only memory), solid-state drives (SSDs) (includingother solid-state storage such as solid state hybrid drives (SSHDs)),hard disk drives (HDDs), magnetic or optical disc drives, or othersuitable memory.

Traffic signal system 20 can also include one or more network interfaces(including any suitable hardware, such as network interface cards (NIC)or wireless NIC (WNIC)) and may be able to communicate with one or moreremote servers via land network 14. Also, traffic signal system 20 caninclude other network capabilities, such as cellular or other wirelesscommunication capabilities, such as is indicated for RSE 26. The trafficsignal can be send traffic signal state information or data to a remotefacility, such as remote facility 16 or municipal facility 18. Thetraffic signal state information or data can include the traffic lightsignal phase of a traffic signal, such as whether a traffic light isGREEN, RED, or YELLOW (or AMBER). The traffic light signal phase andother traffic signal state data can include or be accompanied by aunique identifier (ID) that is used to identify the particularintersection at which the traffic signal is located, as well as atimestamp (or other time indicator) that indicates the time associatedwith the traffic signal data. The traffic signal state data can includea traffic light signal phase P_(t), and a traffic light signal timingT_(t), and/or other signal phase and timing (SPaT) data. Also, any ofthis traffic signal state data can include or be accompanied by timedata that provides a time or a range of times in which the signal statusrefers to and/or is associated with, such as the times in which thesignal will be of a certain status. Although only one traffic signalsystem 20 is depicted, numerous traffic signal systems may be used andeach may include one or more traffic signals 22 or other trafficsignaling devices.

Traffic signal 22 is depicted as a stoplight or traffic light (“R” forRED phase or light, “Y” for YELLOW or AMBER (collectively, referred toas “YELLOW”) phase or light, and “G” for GREEN phase or light), but itshould be appreciated that other traffic signaling devices can be usedinstead, such as any electronic signaling device that may be used todirect vehicular traffic. Additionally, although there is only onetraffic signal shown, it should be appreciated that numerous trafficsignals may be used in system 1 and/or traffic signal system 20, andthat various types of traffic signaling devices may be used.

RSE 26 can be controlled by controller 24 and may include an inductanceloop detector, a video detector, or other traffic-related equipmentand/or sensors that may be situated along a roadside or near anintersection. The RSE 26 and/or controller 24 can include networkcommunication interfaces, such as WNIC or other NIC, and may communicatedirectly with one or more nearby vehicles, such as via short-rangewireless communications (SRWC). Both the RSE 26 and the traffic signal22 may be remotely controlled, and may be reprogrammed or reconfiguredsuch that the operation of signaling cues of the traffic signal isupdated. For example, municipal facility 18 may send a set ofinstructions that can be used to update the traffic signal 22. The setof instructions can be referred to as an “update” and may be sent vialand network 14, one or more cellular carrier systems 13, or other radiosignals. In other embodiments, the traffic signal may be reprogrammedthrough a controller located at or near the traffic signal and that isconnected to the traffic signal via a bus or other communication means.

Vehicles 10,11,12 are each depicted in the illustrated embodiment as apassenger car, but it should be appreciated that any other vehicleincluding motorcycles, trucks, sports utility vehicles (SUVs),recreational vehicles (RVs), bicycles, other vehicles or mobilitydevices that can be used on a roadway or sidewalk, etc., can also beused. As depicted in the illustrated embodiment, the host vehicle 10includes a GNSS receiver 32 with its antenna 34 to receive GNSS radiosignals from satellites 50. The host vehicle 10 further includes atelematics unit 40 that enables communication between the vehicle andremote servers or network devices, such as those at remote facility 16.The GNSS receiver 32 may be any suitable commercially available GNSSreceiver and may provide National Marine Electronics Association (NMEA)or other output messages to telematics unit 40, which may then be sentas GNSS information from the host vehicle 10. According to at least someembodiments, such as where the front vehicle 11 is a CV, the frontvehicle 11 includes the same components, devices, and modules as thehost vehicle 10, even though these components are not separatelydepicted or discussed. According to at least some embodiments, the AV 12includes the same components, devices, and modules as the host vehicle10, even though these components are not separately depicted ordiscussed.

The global navigation satellite system (GNSS) receiver 32 receives radiosignals from the constellation of GNSS satellites 50. The GNSS receiver32 can then obtain GNSS data that provides a location and/or a time.Additionally, the GNSS receiver 32 may be used to provide navigation andother position-related services to the vehicle operator. The GNSS datacan be supplied to remote facility 16 or other remote facility, such asmunicipal facility 18, for certain purposes, such as for vehicletrajectory prediction or other traffic-related purposes. In someembodiments, the GNSS receiver 32 may be a global positioning system(GPS) module that receives GPS signals from GPS satellites that are apart of the U.S. GPS satellite system. Receivers for use with GLONASSand/or Europe's Galileo system may also be used. The GNSS signals may beused to generate GNSS data that includes time data, and this time datamay be the time when the GNSS module receives information fromsatellites 50, a time indicated in the GNSS signals received from theGNSS satellites 50, or other contemporaneous timestamp. Although theillustrated embodiment is described primarily as it would be implementedwith connected vehicles (CVs) that utilize a global navigation satellitesystem (GNSS) for position information, it will be appreciated as thedescription proceeds that the system and methods discussed herein may beused with other WCDs such as a handheld wireless device having a GNSSreceiver along with cellular and/or short range wireless communication(SRWC) capabilities.

In one embodiment, the host vehicle 10, the front vehicle 11, and/or theAV 12 may be configured to periodically record GNSS data and/orperiodically send GNSS data to the remote facility 16. For example, thevehicle may record this GNSS data at a frequency of 10 Hz and send thisinformation to the remote facility at a frequency of 1 Hz—of course,other sampling/recording and/or sending frequencies can be used.Additionally, the vehicle may send heading information (e.g., thedirection the front of the vehicle is facing) or other vehicle or WCDinformation to the remote facility 16. As discussed above, once theremote facility 16 receives GNSS data from the host vehicle 10, thefront vehicle 11, and/or the AV 12, the remote facility 16 may store theinformation at a memory device and/or may process the data according toone or more set of computer instructions, such as computer instructionsthat may be configured to carry out at least part of the methoddescribed herein.

Telematics unit 40 is an electronic control and includes a cellularchipset 42, a processor 44, a memory 46, and an antenna 48. Telematicsunit 40 can be an OEM-installed (embedded) or aftermarket device that isinstalled in the vehicle and that enables wireless voice and/or datacommunication over wireless carrier system 13 and via wirelessnetworking. This enables the vehicle to communicate with remote facility16 and/or municipal facility 18, other telematics-enabled vehicles, orsome other entity or device. The telematics unit preferably uses radiotransmissions to establish a communications channel (a voice channeland/or a data channel) with wireless carrier system 13 so that voiceand/or data transmissions can be sent and received over the channel. Thetelematics unit 40 can receive GNSS data from GNSS receiver 32 and cansubsequently send the GNSS data to remote facility 16 or municipalfacility 18. The telematics unit 40 may be connected to an intra-vehiclecommunications bus 30 that enables communication with other electronicsystems on the vehicle. In some embodiments, the telematics unit 40 canprovide both voice and data communication, and/or can enable the vehicleto offer a number of different services including those related tonavigation, telephony, emergency assistance, diagnostics, infotainment,etc. Data can be sent either via a data connection, such as via packetdata transmission over a data channel, or via a voice channel usingtechniques known in the art.

According to one embodiment, the telematics unit 40 utilizes cellularcommunication according to either GSM, CDMA, or LTE standards and thusincludes a standard cellular chipset 42 for voice communications likehands-free calling, a wireless modem for data transmission, anelectronic processing device or processor 44, one or more digital memorydevices 46, and a dual antenna 48. It should be appreciated that themodem can either be implemented through software that is stored in thetelematics unit and is executed by processor 44, or it can be a separatehardware component located internal or external to telematics unit 40.The modem can operate using any number of different standards orprotocols such as LTE, EVDO, CDMA, GPRS, and EDGE. Wireless networkingbetween the vehicle, RSE 26, and other networked devices can also becarried out using telematics unit 40. For this purpose, telematics unit40 can be configured to communicate wirelessly according to one or morewireless protocols, including short range wireless communication (SRWC)such as any of the IEEE 802.11 protocols, V2I communication protocols,V2V communication protocols, Wi-Fi™, WiMAX™, ZigBee™, Wi-Fi Direct™,Bluetooth™, or near field communication (NFC). When used forpacket-switched data communication such as TCP/IP, the telematics unitcan be configured with a static IP address or can set up toautomatically receive an assigned IP address from another device on thenetwork such as a router or from a network address server.

The processor 44 can be any type of device capable of processingelectronic instructions including microprocessors, microcontrollers,host processors, controllers, vehicle communication processors, andapplication specific integrated circuits (ASICs). It can be a dedicatedprocessor used only for telematics unit 40 or can be shared with othervehicle systems. The processor 44 executes various types ofdigitally-stored instructions, such as software or firmware programsstored in memory 46, which enable the telematics unit 40 to provide awide variety of services. For instance, the processor 44 can executeprograms or process data to carry out at least a part of the methoddiscussed herein. In one embodiment, the telematics unit 40 includes anapplication (e.g., computer program) that enables the processor to sendGNSS data to a remote facility 16. The memory 46 can include poweredtemporary memory and/or any suitable non-transitory, computer-readablemedium such as different types of RAM (random-access memory, includingvarious types of dynamic RAM (DRAM) and static RAM (SRAM)), ROM(read-only memory), solid-state drives (SSDs) (including othersolid-state storage such as solid state hybrid drives (SSHDs)), harddisk drives (HDDs), magnetic or optical disc drives, or other suitablememory.

Furthermore, it should be understood that at least some of theaforementioned modules could be implemented in the form of softwareinstructions saved internal or external to the telematics unit 40, theycould be hardware components located internal or external to thetelematics unit 40, or they could be integrated and/or shared with eachother or with other systems located throughout the vehicle, to cite buta few possibilities. In the event that the modules are implemented asVSMs located external to the telematics unit 40, they could utilize thevehicle bus 30 to exchange data and commands with the telematics unit.

The mobile device 60 is a device that can communicate with other devicesusing cellular carrier system 13 and/or land network 14. The mobiledevice 60 can communicate with remote facility 16 and/or municipalfacility 18, and is thus a wirelessly connected device (WCD).Additionally, mobile device 60 may communicate with vehicles 10,11,12and/or RSE 26 via short-range wireless communications (SRWC), such asBluetooth™, Bluetooth Low Energy™ (BLE), Wi-Fi™, near fieldcommunications (NFC), or various other SRWC. The mobile device 60 mayinclude: hardware, software, and/or firmware enabling such cellulartelecommunications and SRWC as well as other mobile device applications.The hardware of the mobile device 60 may comprise: a processor andmemory (e.g., non-transitory computer readable medium configured tooperate with the processor) for storing the software, firmware, etc. Themobile device may also include a GNSS receiver or module, such as amodule that is similar to GNSS receiver 32 that is included in vehicles10,11,12. The processor and memory may enable various softwareapplications, which may be preinstalled or installed by the user (ormanufacturer). One implementation of a vehicle-mobile device applicationmay enable GNSS information (or trajectory prediction input data) to besent to the remote facility 16. In one embodiment, a mobile device maybe inside a vehicle cabin of a vehicle and, thus, may be used to sendGNSS information or trajectory prediction input data to a remotefacility 16. Thus, a non-connected vehicle (e.g., a vehicle without aGNSS module and/or without capabilities to connect to remote facility 16or municipal facility 18) transporting a cellular device or other WCDoperates as a virtual connected vehicle and, in at least someembodiments, may thus be treated as a connected vehicle (CV).

As mentioned above, an artificial intelligence (AI) vehicle trajectoryprediction model can be used to predict trajectories and/oracceleration/velocity of a human-driven vehicle at a particular time orseries of time. The AI vehicle trajectory prediction model incorporatestraffic signal phase and timing (SPaT) information, which includestraffic light signal phase P_(t) and traffic light signal timing T_(t),as well as the host vehicle-traffic light distance d_(x) and thelongitudinal host vehicle speed v_(x). The time of day (TOD) is alsotaken into consideration since the traffic characteristics includingcongestion and speed may differ considerably depending on the time ofday. The state input for time t can be denoted as X_(t) and representedas follows:X _(t):=[X _(t) ^(HV) ,X _(t) ^(FV) ,X _(t) ^(TL) ,TOD _(t)]where X^(HV) represents the host vehicle state (which, in the presentembodiment, includes the host vehicle-traffic light distance d_(x) andthe longitudinal host vehicle speed v_(x)), X^(FV) represents the frontvehicle state (which, in the present embodiment, includes the front-hostvehicle distance r_(t) and the front-host vehicle speed {dot over(r)}_(t)), X^(TL) represents the traffic light state (which, in thepresent embodiment, includes traffic light signal phase P_(t) andtraffic light signal timing T_(t)), and TOD represents the time of day.It should be appreciated that the subscript t indicates that the time ofeach of the respective states or values correspond to the present timet. It should be appreciated that, in some embodiments, the host vehiclestate X^(HV) includes a series or sequence of values corresponding toone or more times prior to the present time and which can be defined ast-τ. The AI vehicle trajectory prediction model maps the state input Xto a predicted trajectory, which includes a longitudinal acceleration ofthe vehicle (at least in many embodiments). Each of these five inputs istaken with respect to a particular vehicle approaching a particulartraffic signal.

Host Vehicle-Traffic Light Distance d_(x) represents a longitudinaldistance between the host vehicle and the traffic light that the vehicleis approaching or departing from. This distance d_(x) can be useful inpredicting the behavior of the human-driven vehicle. For example, avehicle approaching a traffic light in the RED phase typically travelsrelatively slowly and slows down when it is close to the traffic light,whereas the vehicle may travel fast when the vehicle is far away fromthe traffic light. When the host vehicle-traffic light distance d_(x) iszero (i.e., d_(x)=0), then this represents that the host vehicle is atthe stop line of a lane that is subject to the traffic light. When thehost vehicle-traffic light distance d_(x) is greater than zero (i.e.,d_(x)>0), this means that the host vehicle is approaching the trafficlight (upstream) and, when the host vehicle-traffic light distance d_(x)is less than zero (i.e., d_(x)<0), this means that the host vehicle isdeparting from the traffic light (downstream).

Longitudinal Host Vehicle Speed v_(x) indicates a longitudinal speed ofthe vehicle. This speed v_(x) can be useful in predicting the behaviorof the human-driven vehicle. For instance, in the absence of a RED orYELLOW traffic light, a vehicle traveling at a relatively low speedcompared to the speed of the surrounding traffic is more likely toaccelerate. Another example is that a vehicle approaching a trafficlight in the RED phase at a high speed tends to break harder (i.e., usethe brakes to decelerate the vehicle faster) than a vehicle approachingthe traffic light at a low speed. The longitudinal host vehicle speed isassumed to be greater or equal to zero (i.e., v_(x)>=0). Thelongitudinal host vehicle speed v_(x) is the speed of the human-drivenvehicle as taken along the longitudinal direction L_(x) (FIG. 4 ).

Front-Host Vehicle Distance r_(t) indicates a longitudinal distancebetween the host vehicle and the front vehicle. This distance r_(t) maybe measured as the longitudinal distance between any two referencepoints on the host vehicle and the front vehicle. In one embodiment, thetwo reference points can be a point at the centers of the host vehicleand the front vehicle as taken in the longitudinal direction. In anotherembodiment, the two reference points can be a front-most reference pointof the host vehicle (e.g., a point on the front bumper of the hostvehicle) and a rear-most point of the front vehicle (e.g., a point onthe rear bumper of the front vehicle). It is assumed that the positon ofthe front vehicle, which can be denoted as r_(x) is greater than 0,meaning that the front vehicle is ahead of the host vehicle since theposition is taken with respect to the host vehicle (i.e., the hostvehicle is always at the position of 0).

Front-Host Vehicle Speed {dot over (r)}_(t) is the longitudinal speed ofthe front vehicle with respect to the longitudinal speed of the hostvehicle, which can be determined, for example, based on determining thedifference between the absolute longitudinal speed (i.e., thelongitudinal speed with respect to earth) of the front vehicle and theabsolute longitudinal speed of the host vehicle.

Traffic Light Signal Phase P_(t) represents the phase of the trafficlight that the vehicle is subject to (i.e., that the vehicle isapproaching). In the examples discussed herein, the GREEN, YELLOW, andRED phases are each represented by the integers 1, 2, and 4,respectively. Needless to say, the driver behavior at a traffic light inthe GREEN phase is different from that at a traffic light in the YELLOWor RED phase. In other embodiments, the traffic light or other signalingdevice may have other phases, such as “BLINKING YELLOW”, and themethod/system can be modified to take into account these different oradditional phases.

Traffic Light Signal Timing T_(t) is an amount of time relative to thelast (or next) phase change of the traffic signal. In the presentembodiment, the traffic light signal timing T_(t) is the amount of timethat has elapsed since the last phase change of the traffic signal.Examples of phase changes of a traffic signal would be the trafficsignal transitioning from the GREEN phase to the YELLOW phase, from theYELLOW phase to the RED phase, or from the RED phase to the GREEN phase.Every time a phase transition occurs, T_(t) is initialized to zero (0)and is accumulated as time elapses. This traffic light signal timingT_(t) may account for transient behaviors of human drivers near trafficsignal phase changes. For example, a vehicle approaching an intersectionin a RED phase with a small T_(t) (meaning that the phase has justshifted to red) may not be traveling slow whereas a vehicle approachingan intersection in a RED phase with a large T_(t) is likely to travelslow or stop (or already be stopped). Another example is when a vehicleis stationary in a queue and a phase shift from the RED phase to theGREEN phase recently occurred. In such a scenario, depending on theposition of the vehicle in the queue, the vehicle may or may not staystationary for a while. Thus, T_(t) can be an important factor forpredicting driver behaviors and, thus, the human-driven vehicletrajectory. In at least some embodiments, the traffic light signaltiming T_(t) can be used to indirectly account for the queue formed neartraffic lights.

Time of Day TOD represents the time of day as elapsed time since thebeginning of the day. In one embodiment, the TOD can represent thenumber of hours that have elapsed since the beginning of the day (i.e.,0<=TOD<24). When the TOD=0, then it can be said that the time of day ismidnight, and when the TOD=12, it can be said that the time of day isnoon. The traffic characteristics, including congestion and speed, maydiffer considerably depending on the time of day; for example, trafficspeed may be much slower during rush-hour than at other times such as inthe middle of the night.

Due to the stochastic and complex nature of human decision making indriving, a simple analytical model such as a linear or a physics-basedmodel may not be suitable or preferable for accurately representing thenominal or probabilistic behaviors of human-drivers near trafficsignals. Instead, an improved model can be learned through data-drivenmodeling, using deep learning methods on historical driving data. Thelearned proposed models can be either deterministic or probabilistic.

With reference to FIGS. 2A-2B, there is shown an example of graphsdepicting the host vehicle position relative to the position of atraffic signal and the host vehicle speed (relative to earth). Thisexample is based sampled real-world data and demonstrates howuncertainty of traffic light phases and timing makes it difficult toaccurately predict a trajectory of a human-driven vehicle. FIG. 2Adepicts possible predictions that are made without knowing the phase andtiming of the traffic light and FIG. 2B depicts the actual(ground-truth) distance relative to the traffic light and the actuallongitudinal host vehicle speed (relative to earth). The sample is a 7 s(7 second) long red-green (RG) scenario, where the present time is t₀and τ=2 s and where the prediction window is 5 s. The host vehicle stateX^(HV) thus includes, in this example, host vehicle states from the lasttwo seconds from t₀-τ to t₀ and the ground-truth future vehicle statesfrom t₀ and t₀+5 s as shown by the black lines indicated at 102 a, 102b. As indicated by the solid horizontal bar 104 from time t=−2 to t=0,the phase of the traffic signal is red. The bar 104 is at a distance of13 meters (13 m) from the host vehicle as shown in the top graph of FIG.2A. It should be appreciated that the distance is initialized to 0 mbased on the starting position at time t₀-τ (or −2 s in this example).The phase of the traffic signal after time t=0 is not depicted in FIG.2A since the host vehicle is unaware of what the phase timing is for thetraffic signal. Thus, at time t=0 (or time t₀), provided the hostvehicle's position (0 m) and speed (0 m/s), and the uncertainty of whenthe traffic signal will change from the red phase to the green phase, areasonable prediction that an existing method would make is to predictthat the vehicle remains in the same position (as indicated at 106) andat a speed of 0 m/s (as indicated at 108). However, in reality of thisexample as shown in FIG. 2B, the phase shifted to green at t=0.35 s (asindicated by the bar 104 at distance=13 m), and the vehicle increasedits speed (as indicated at 102 b) and traveled toward the traffic signal(as indicated at 102 a) as depicted black lines.

With reference to FIG. 3 , there is shown an exemplary scenario in whichthe human-driven host vehicle 10 is approaching the traffic signal 22,which will proceed through the intersection at which the traffic signalis located—that is, continue on the same road as the vehicle passesthrough the intersection. Also, as is shown in FIG. 3 , the frontvehicle 11 is presently in front of the host vehicle 10. Moreover, theAV 12 may be located behind the HV (on the other side of the HV than theFV) or at another nearby position (in an adjacent lane), although thisis not shown in FIG. 3 . As shown in FIG. 4 , a first exemplary scenario(“Scenario G”) is shown in which the prediction window starts on (orduring) (indicated by t₀) a GREEN light and ends on the same GREEN (“G”)light (indicated by t_(T)); a second exemplary scenario (“Scenario YR”)is shown in which the prediction window starts on (or during) (indicatedby t₀) a YELLOW (“Y”) light and ends on the next RED (“R”) light(indicated by t_(T)); and a third exemplary scenario (“Scenario GYR”) isshown in which the prediction window starts on (or during) (indicated byt₀) a GREEN light and ends on the next RED light (indicated by t_(T)).

There are other scenarios that are not specifically illustrated in FIGS.3-4 , but the principles discussed with respect to those three exemplaryscenarios can be applied to these other scenarios as well and asappreciated by those skilled in the art. There are six base scenarios:G, Y, R, GY, YR, RG. An extended scenario GYR is also discussed, whichcan be used to evaluate the long-term performances of the model. Thefirst letter of each base (or extended) scenario represents the trafficlight phase at the beginning of the prediction window and the lastletter of each base (or extended) scenario represents the traffic lightphase at the end of the prediction window. The prediction window is thetime from t=t₀-τ to t₀ (where t₀ is the present time). The host vehiclestate X_(t) ^(HV), the context vector C_(t)=[X_(t) ^(FV), X_(t) ^(TL),TOD_(t)], and/or one or more parts thereof are obtained during theprediction window. This information is then used to obtain a trajectoryprediction using the AI vehicle trajectory prediction application. Thatis, in at least some embodiments, host vehicle state X^(HV) (or partsthereof), including (for example), the longitudinal host vehicle speedv_(x) and the host vehicle-traffic light distance d_(x), is obtained aplurality of times (e.g., at a set interval) during the predictionwindow (i.e., from time t₀-τ to t₀). Additionally or alternatively, thefront vehicle state X^(FV) (or parts thereof) is obtained during theprediction window. The number of samples to be obtained during theprediction window and/or the length or scope of the prediction windowcan be selected based on the particular implementation or application inwhich the method/system is being used. Moreover, in some embodiments,the host vehicle state X^(HV) may be associated with a host vehicleprediction window and the front vehicle state X^(FV) may be associatedwith a front vehicle prediction window; these windows can be set to thesame or different time periods, to the same or different sampling rate,etc., which can be adjusted based for the particular application orimplementation in which it is used. In some embodiments, the predictionwindow(s) can be set based on an area or distance from the traffic lightor intersection. And, in yet another embodiment, the predictionwindow(s) can each be tailored for each intersection or traffic signal,and may take into account one or more factors, such as the speed limitof the road, the number of lanes of the road, the length of the trafficsignal phases, the phase timings of the traffic signal, etc.

As shown in FIG. 3 , various embodiments discussed herein seek to obtainfuture longitudinal positions and velocities of a host vehicle as afunction of time. This problem can be made challenging due to stochasticmotions of vehicles near traffic signals. For example, a driver mayprefer hard-breaking when the driver approach a red light, while otherdriver prefers soft-breaking. As another example, a driver may prefer toaccelerate hard in a departure scenario, while other drivers prefersoft-departure. Additionally, the reactions of drivers at phasetransitions (G to Y, Y to R, or R to G) are different from those atsteady phases (G, Y, or R). Another example is where a driver approachesa traffic light at a high speed, and the driver is faced with twocompeting decisions—that is, to either make a sudden stop or passthrough the intersection at which the traffic light is located. In thissense, the problem can be broken down into seven distinct scenarios,three of which are presented in FIG. 4 . This breakdown orcategorization of the different scenarios is based on the propositionthat humans react differently at different traffic phases and timings oftraffic signals, which results in the trajectories being significantlydifferent depending on the scenario. For example, a trajectory of ahuman driver at a GREEN light or phase would be notably different fromthat at a RED light or phase.

As mentioned above, the AI vehicle trajectory prediction model (orpolicy, referred to herein collectively “model”) as was trained usingneural networks for a deterministic model (or a model that predicts themost-probable trajectories), and mixture density networks (MDN) for aprobabilistic policy. It was shown that the baseline deterministicneural net model (which utilizes SPaT information) performed farsuperior than existing methods (which utilizes SPaT information) on atest set of training data. In at least some embodiments, theprobabilistic model provides contexts on the stochastic nature of humandriving and measures how confident one can be in the predictions. Insome embodiments, the probabilistic model is capable of learningmulti-modal distributions, and thus can accurately capture two competinghuman policies (pass or stop) in the yellow-light dilemma zone. For thetraining, validation, and testing, 502,253 samples that 50 distinctvehicles have reported over a span of 27 months (March 2015-July 2017)at a particular section of a road with a signalized intersection (SI).Each vehicle reported its 10 Hz GPS signals (coordinates, speeds, andheading angles), which then were used to calculate X_(t) ^(HV). Thetraffic light (TL) profiles X_(t) ^(TL) were obtained from avehicle-to-infrastructure (V2I) communication device installed at theSI, such as the RSE 26. Cameras that were mounted on the front of thehost vehicle and facing the area in front of the host vehicle were usedto obtain the front vehicle state X_(t) ^(FV). In order to reduce noisein X_(t) ^(HV), X_(t) ^(TL), and X_(t) ^(FV), a least-square polynomialsmoothing filter was used, such as that discussed in R. W. Schafer etal., “What is a savitzky-golay filter,” IEEE Signal processing magazine,vol. 28, no. 4, pp. 111-117, 2011.

With reference to FIGS. 5A-5B, there is shown two embodiments of an AIvehicle trajectory prediction model. FIG. 5A depicts an AI vehicletrajectory prediction model that uses a deterministic policy (ƒ_(d):X(t)→a_(x)(t)), which is obtained by training a neural network on thetraining data, for example. FIG. 5B depicts an AI vehicle trajectoryprediction model that uses a probabilistic policy(ƒ_(p):X(t)→p(a_(x)(t)|X(t))), which is obtained by training a mixturedensity network (MDN) on the training data. For example, thedistribution for the MDN can be a Gaussian mixture. Both of these modelsuse a host vehicle state X^(HV) and a context vector C as input. In theembodiment shown, both the deterministic and probabilistic models use adouble stacked-LSTM that takes the host vehicle state X_(t-τ) ^(HV)followed by a concatenation with the context vector C_(t)=[X_(t) ^(FV),X_(t) ^(TL), TOD_(t)]. LSTM refers to a long short-term memory (LSTM),which is an artificial recurrent neural network (RNN) architecture.

The concatenated tensor is then fed into a multi-layer perceptron (MLP)for the deterministic model and a MDN for the probabilistic model. TheMLP layer outputs at a_(t) ^(HV) whereas the MDN layer outputs thedistribution parameters Z_(t). When Gaussian mixtures are used as themixture network, the MDN layer outputs the following three parametersets of the gaussian mixture: mixture weights π_(k), mean of componentsμ_(k), variance of components σ_(k) for k=1, 2, . . . , N and N is thenumber of components. We used N=2 for the yellow light dilemma scenario.Both models were trained on the same data using ADAM optimizer. ƒ_(d) islearned by minimizing a loss function L_(d) which is a summation of meansquared error as described below.L _(d):=Σ_(t=1) ^(T)(a _(t) ^(HV)−ƒ_(d)(X _(t-τ:t) ^(HV) ,C_(t)))^(z)  (Equation 1)ƒ_(p) is obtained by minimizing a loss function L_(p) which is a sum ofa negative log likelihood.L _(p):=Σ_(t=1) ^(T)−log(p(a _(t) ^(HV) |X _(t-τ:t) ^(HV) ,C _(t) ;Z_(t)))  (Equation 2)

According to at least some embodiments, the AI vehicle trajectoryprediction model uses a plurality of hidden layers as a part of mappingthe input to an output. In the AI vehicle trajectory prediction modelthat uses the deterministic policy (or “deterministic prediction model”for short) (FIG. 5A), the output is a predicted acceleration a_(x) ofthe vehicle. In the AI vehicle trajectory prediction model that uses theprobabilistic policy (or “probabilistic prediction model” for short)(FIG. 5B), the output is composed of distribution parameters, which areused to define a distribution of predicted accelerations of the vehicle.This distribution is then sampled to obtain the predicted accelerationa_(x) of the vehicle.

FIG. 6 illustrates an embodiment of the trajectory prediction frameworkused to obtain the predicted trajectories. The first part of theframework is the off-line learning of the AI vehicle trajectoryprediction model which was described previously. The second part iswhere the trained models are used to obtain trajectory predicts orforecasts over the full prediction horizons. It is an iterative process,where instantaneous predictions and propagations of a vehicle dynamicsare alternated between. The one-step propagation of the longitudinalvehicle dynamics is done by utilizing a zero-order hold for discretedynamics equations described in Equations 3, 4, and 5, which arediscussed below.

$\begin{matrix}{{v_{n + 1}:} = {v_{n} + {a_{n}\Delta t_{n}}}} & ( {{Equation}\mspace{14mu} 3} ) \\{{d_{n + 1}:} = {d_{n} + {{0.5}( {v_{n + 1} + v_{n}} )\Delta t_{n}}}} & ( {{Equation}\mspace{14mu} 4} ) \\{{X_{n + 1}^{HV} = {{A_{n}X_{n}^{HV}} + {B_{n}a_{n}^{HV}\mspace{14mu}{where}}}}{{A_{n}:=\begin{bmatrix}1 & {\Delta t_{n}} \\0 & 1\end{bmatrix}},{B_{n}:=\begin{bmatrix}{{0.5}\Delta t_{n}} \\{\Delta t_{n}}\end{bmatrix}},{X_{n}^{HV} = {\begin{bmatrix}d_{n} \\v_{n}\end{bmatrix}.}}}} & ( {{Equation}\mspace{14mu} 5} )\end{matrix}$

Defining n=0, and n=N as the index for t=0, and t=T, the trajectoryprediction over the prediction horizon t=[0, T] are obtained bypropagating Equation 5 from n=0 to n=N−1:

$\begin{matrix}{X_{N}^{HV} = {{{\prod\limits_{k = 0}^{N - 1}\;{A_{k}X_{0}^{HV}}} + {\prod\limits_{k = 1}^{N - 1}\;{A_{k}B_{0}a_{0}^{HV}}} + {\prod\limits_{k = 2}^{N - 1}\;{A_{k}B_{1}a_{1}^{HV}}} + \ldots + {A_{N - 1}B_{N - 2}a_{N - 2}^{HV}} + {B_{N - 1}a_{N - 1}^{HV}}} = {{{\prod\limits_{k = 0}^{N - 1}\;{A_{k}X_{0}^{HV}}} + {\prod\limits_{k = 1}^{N - 1}\;{A_{k}B_{0}{f( {X_{{{- n_{\tau}} + 1}:0}^{HV},C_{0}} )}}} + \ldots + {B_{N - 1}{f( {X_{N - {n_{\tau}:{N - 1}}}^{HV},C_{N - 1}} )}}}:={F( {X_{1:{N - 1}}^{HV},X_{{{- n_{\tau}} + 1}:0}^{HV},C_{0:{N - 1}},{\Delta\; t_{0:{N - 1}}}} )}}}} & ( {{Equation}\mspace{14mu} 6} )\end{matrix}$where ƒ can either be ƒ_(d) or S(ƒ_(p)). S is a function which returns asample at a_(t) ^(HV) according to the probability density function(pdf) of Z_(t) (In case of an uni-modal gaussian distribution,Z_(t)=[μ_(t), σ_(t)] and a_(t) ^(HV)˜N(μ_(t), σ_(t) ²)). τ, n_(τ) eachindicates input sequence length in time, and in the number of steps,respectively.

As described in Equation 6, X_(N) ^(HV) is a function (F) of [X_(1:N-1)^(HV), X_(−n) _(τ) _(+1:0) ^(HV),C_(0:N-1),Δt_(0:N-1)]. The second termX_(−n) _(τ) _(+1:0) ^(HV) is obtained at t=0, and the last termΔt_(0:N-1) can simply be predetermined at the prediction time based on arequired time resolution. Obtaining C_(0:N-1) at the prediction time(t=0) is the main challenge, due to uncertainties in X_(1:N-1) ^(FV),X_(1:N-1) ^(TL). A simple way to get away with the uncertainties is tosimply design the model to predict trajectories (X_(0:T) ^(HV))conditioned on only the observed states [X_(−τ:0) ^(FV), X_(−τ:0)^(TL)]. An example is a model with many-to-many RNN that takes asequence of past states and returns a sequence of future states.However, this approach does not solve the prediction problem that wasdiscussed in the previous sections and FIGS. 2A-2B.

As briefly mentioned, to solve such a prediction problem (and otherprediction/forecasting problems), uncertainties of human-driving can beremoved by utilizing the future phases and timings of TLs obtainedthrough, for example, vehicle-to-infrastructure communications. With theaccess to the future states of TLs, X_(1:N-1) ^(TL) can be obtained atthe prediction time. In this embodiment, the remaining input to be usedis then X_(1:N-1) ^(FV), which is predicted based on a variant of thehuman policy model ƒ_(d). Specifically, another human policy model(ƒ_(d) ^(NoFV)) is trained: ƒ_(d) ^(NoFV): [X_(N-n) _(τ) _(:N)^(HV),C′_(N)]→a_(N) ^(HV) with C′_(N):=[X_(N) ^(TL), TOD_(N)] takingX_(N) ^(FV) out of the context. After the off-line learning of ƒ_(d)^(NoFV) is done, the iterative process described early in this sectionis applied on FV to obtain X_(1:N-1) ^(FV) via Equation 6 with ƒ_(d)^(NoFV). Once [X_(1:N-1) ^(FV), X_(1:N-1) ^(TL)] are calculated, theresulting trajectory prediction X_(1:N) ^(HV) can be obtained. Note,X_(1:N) ^(HV) can simply be predicted using ƒ_(d) ^(NoFV) as well. Anablation study was conducted on ƒ_(d), ƒ_(d) ^(NoFV) and other twomodels (ƒ_(d) ^(NoTL),ƒ_(d) ^(NoFVTL)) which each represents a modelwhere X^(TL) and [X^(FV),X^(TL)] are taken out of the context,respectively.

For the probabilistic human policy model, the probability of theresulting trajectory prediction p(X_(1:N) ^(HV)) can be estimated usingthe chain rule of probability, which factorizes the joint distributionover N separate conditional probabilities:

$\begin{matrix}{{p( { X_{1:N}^{HV} \middle| X_{{{- n_{\tau}} + 1}:0}^{HV} ,C_{0:{k - 1}},{\Delta\; t_{0:{k - 1}}}} )} = {\prod\limits_{k = 1}^{N}{p( { X_{k}^{HV} \middle| X_{1:{k - 1}}^{HV} ,X_{{{- n_{\tau}} + 1}:0}^{HV},C_{0:{k - 1}},{\Delta\; t_{0:{k - 1}}}} )}}} & ( {{Equation}\mspace{14mu} 7} )\end{matrix}$As opposed to the deterministic predicting where the most-probabletrajectory is obtained, a resulting trajectory is sampled from aprobability distribution. While the probability of a trajectoryprediction can be estimated via Equation 7, the probability densityfunction for X_(t) ^(HV) needs to be numerically estimated since thedistribution parameter Z_(t) is obtained via an arbitrarily complexneural network. Thus, Monte Carlo Simulation (see R. Y. Rubinstein andD. P. Kroese, Simulation and the Monte Carlo method. John Wiley & Sons,2016, vol. 10) is utilized to obtain the samples (roll-out trajectories)and kernel density estimation is utilized to approximate the probabilitydensity functions of the samples.

With reference to FIG. 7 , there is shown an embodiment of a method 200for obtaining a predicted trajectory of a human-driven host vehicle asthe human-driven host vehicle approaches a traffic signal. In at leastone embodiment, the method 200 is carried out by the host vehicle. Inanother embodiment, the method 200 is carried out by another vehicle,such as the AV 12 or the front vehicle 11, that is approaching thetraffic signal, or by another vehicle and the host vehicle. In otherembodiments, the method 200 is carried out by a traffic signal systemand/or one or more remote servers, which can be located at one or moreremote facilities. And, in some embodiments, the method 200 is carriedout by a combination of one or more vehicles (e.g., the autonomousvehicle 12, the host vehicle 10, and/or the front vehicle 11) and one ormore remote servers—that is, one or more steps can be carried out by oneor more vehicles and one or more steps can be carried out by the one ormore remote servers. Or, in other embodiments, the method 200 is carriedout by a combination of one or more vehicles and the traffic signalsystem, or by a combination of the one or more vehicles, the trafficsignal system, and the one or more remote servers. In the discussion ofthe method 200 below, the trajectory is being predicted for the hostvehicle 10 as the host vehicle 10 approaches (or is near or at) thetraffic signal 22. In the scenario discussed below, the front vehicle 11is another vehicle that is approaching (or has approached) (or isotherwise near or at) the traffic signal 22 and in front of the hostvehicle. In some scenarios, there may be no front vehicle (or vehicle infront of the host vehicle) at the intersection and, thus, the method 200may be adapted to address such scenarios, as will be discussed below.While the steps of the method 200 are described as being carried out ina particular order below, it is hereby contemplated that the method 200can be carried out in any technically feasible order as will beappreciated by those skilled in the art.

As mentioned above, the method 200 is used to obtain a predictedtrajectory of a human-driven host vehicle (e.g., the host vehicle 10) asthe human-driven host vehicle approaches a traffic signal. In oneembodiment, the method 200 is carried out by the AV 12 that is separatefrom the host vehicle 10 (i.e., the AV and the host vehicle aredifferent vehicles) and that is approaching the traffic signal, or thatis otherwise nearby. In at least some of such embodiments where themethod 200 is carried out by the AV 12, the AV 12 can obtain the hostvehicle state X^(HV) from the host vehicle by use of V2V communicationsor by V2I communications between the host vehicle 10 and the trafficsignal system 20 and V2I communications between the AV 12 and thetraffic signal system 20. The method 200 may be carried out by one ormore electronic controllers having at least one processor and memory.For example, in one embodiment the method 200 is carried out by thetelematics unit of the AV 12. In another embodiment, the method 200 iscarried out by one or more remote servers 17 of the remote facility 16and/or the electronic controller 24 of the traffic signal system 20.And, in yet another embodiment, the method 200 is carried out by acombination of the telematics unit of the AV 12 and one or more remoteservers 17.

The method 200 can be initiated in a number of different ways. In oneembodiment where the method 200 is carried out by the host vehicle, themethod 200 can be initiated by the host vehicle in response todetermining that the host vehicle is approaching a traffic signal. Forexample, the GNSS data of the host vehicle can be used along withgeographical/roadway map data to determine that the host vehicle is nearand/or approaching a traffic signal. In response, the method 200 canbegin with step 210. In another embodiment, the host vehicle canperiodically send its GNSS data to a remote server, such as those of theone or more remote facilities. The remote facility can then use the GNSSdata along with geographical/roadway map data to determine that the hostvehicle is near and/or approaching a traffic signal. In addition, thehost vehicle and/or the remote facility can determine that anothervehicle is at or approaching the same traffic signal, such as throughusing GNSS data of that vehicle or other information. In addition, itcan be determined that this other vehicle is a human-driven vehiclebased on receiving data from the vehicle indicating that it is ahuman-driven vehicle. In another embodiment, such as those in which theAV carries out the method 200 (or part(s) thereof), the method 200 canbe initiated by the AV detecting the presence of a host vehicle and/ordetecting that the host vehicle is a human-driven vehicle.

The method 200 begins with step 210, where a host vehicle state X_(t)^(HV) is obtained, which includes (at least in the present embodiment) ahost vehicle-traffic light distance d_(x) and a longitudinal hostvehicle speed v_(x). In at least one embodiment, the hostvehicle-traffic light distance d_(x) and the longitudinal host vehiclespeed v_(x) (or information used to determine these values, which isreferred to as “host vehicle base information” (e.g., a GNSS location ofthe host vehicle 10 used for determine the distance d_(x))) are obtainedat the host vehicle 10. In one embodiment, the GNSS receiver 32 of thehost vehicle 10 receives a plurality of GNSS signals (e.g., GPS signals)and then determines a GNSS location as well as a time based on thesesignals. This GNSS location is then used to obtain the hostvehicle-traffic light distance d_(x). For example, as mentioned above,the vehicle may have stored geographical/roadway map data that indicatesa geographical location (e.g., geographical coordinates) of a referencepoint corresponding to the traffic signal and/or intersection (referredto as a “traffic signal geographical position”). The traffic signalgeographical position is then used along with the GNSS location todetermine the host vehicle-traffic light distance d_(x). Thelongitudinal host vehicle speed v_(x) may also be determined based onthe GNSS data that is obtained from the GNSS signals.

In some embodiments, the host vehicle state X_(t) ^(HV) is determined orotherwise obtained at the traffic signal system 20 through the use ofone or more sensors of the RSE 26 (or of the traffic signal system 20),such as a video detector, other image or electromagnetic sensor, orinductance loop sensor, or by one or more onboard sensors of anothernearby vehicle (e.g., a nearby autonomous vehicle). Additionally oralternatively, the host vehicle-traffic light distance d_(x), thelongitudinal host vehicle speed v_(x), or any host vehicle baseinformation thereof can be determined or otherwise obtained by thetraffic signal system 20 through the traffic signal system 20 receivingone or more wireless signals from the host vehicle 10 in embodimentswhere the host vehicle 10 is a connected vehicle (CV). In someembodiments where the host vehicle 10 is a CV, the host vehicle 10 canuse a short-range wireless communications (SRWCs) circuit or chipsetthat enables SRWCs (e.g., Wi-Fi™, Bluetooth™) The host vehicle 10 and/orthe traffic signal system 20 can send the host vehicle state X_(t) ^(HV)to the AV 12 via V2V or V2I communications.

In some embodiments, the host vehicle 10 can include one or moreon-board sensors that are capable of determining the hostvehicle-traffic light distance d_(x), the longitudinal host vehiclespeed v_(x), or any host vehicle base information thereof—for example,the host vehicle 10 can include a digital camera that captures images ofthe host vehicle's surroundings and then uses object recognition (orother techniques) to determine a geographical location of the hostvehicle based on recognizing landmarks in conjunction withgeographical/roadway map data that associates the identifiable landmarkswith a geographical location. This geographical location of the hostvehicle 10 then be used along with a geographical location of thetraffic signal (which can be pre-stored at the remote facility 16 (ormunicipal facility 18)) to determine the host vehicle-traffic lightdistance d_(x). The host vehicle-traffic light distance d_(x), thelongitudinal host vehicle speed v_(x), or any host vehicle baseinformation thereof can be received or otherwise obtained by the one ormore remote servers 17. The remote server(s) 17 can receive the hostvehicle-traffic light distance d_(x) and the longitudinal host vehiclespeed v_(x) from the traffic signal system 20 or from the host vehicle10. It should be appreciated that the host vehicle-traffic lightdistance d_(x) and the longitudinal host vehicle speed v_(x) do notnecessarily have to be obtained in the same manner—for example, the hostvehicle-traffic light distance d_(x) can be obtained from a sensor atthe RSE 26 and then sent to the remote server(s) 17, and thelongitudinal host vehicle speed v_(x) can be determined at the hostvehicle 10 and then sent to the remote server(s) 17 directly (i.e.,without first being sent to the host vehicle 10 and/or the trafficsignal system 20). The method 200 then continues to step 220.

In step 220, a traffic signal state X_(t) ^(TL) is obtained, whichincludes a traffic light signal phase P_(t) and an traffic light signaltiming T_(t) are obtained. According to at least some embodiments, thetraffic signal state X_(t) ^(TL) can include other state informationconcerning the state of the traffic signal. The traffic light signalphase P_(t) and the traffic light signal timing T_(t) (or informationused to determine these values, which is referred to as “traffic signalbase information” (e.g., the times at which the phase of the trafficlight changes)) may be known to the municipal facility 18 and stored ina database of the municipal facility 18. This information can then besent to the remote server(s) 17 and/or to the host vehicle 10.Additionally or alternatively, this information can be stored at thetraffic control system 20 and then sent to the remote server(s) 17 theAV 12, and/or to the host vehicle 10. For example, the traffic controlsystem 20 and/or the municipal facility 18 can store a schedule of timesthat indicates the times at which the traffic light is of a particularphase or signal (e.g., RED, GREEN, YELLOW). In some traffic lightcontrol systems, the output of the traffic signal may be dynamic—forexample, the traffic light may only switch to GREEN for a road when avehicle is detected at the traffic light on the road. In suchembodiments, the traffic control system 20 can record and send thisinformation of the phase change to the host vehicle 10, the municipalfacility 18, and/or the remote server(s) 17. The method 200 continues tostep 230.

In step 230, a front vehicle state X_(t) ^(FV) is obtained, whichincludes (at least in the present embodiment) a front-host vehicledistance r_(t) and a front-host vehicle speed {dot over (r)}_(t). Insome embodiments where the front vehicle 11 is a CV, the front-hostvehicle distance r_(t) and the front-host vehicle speed {dot over(r)}_(t) (or information used to determine these values, which isreferred to as “front vehicle base information” (e.g., a GNSS locationof the front vehicle 11 used for determine the distance r_(t) or speed{dot over (r)}_(t))) can be initially determined or otherwise obtainedat the front vehicle 11, such as through use of a GNSS receiver of thefront vehicle 11. The GNSS data of the front vehicle 11 (or “frontvehicle GNSS data”) can then be sent via SRWC (e.g., V2V communications)to the host vehicle 10. The front vehicle GNSS data is then used alongwith the host vehicle GNSS data (as obtained in step 210, for example)to determine the front-host vehicle distance r_(t) and the front-hostvehicle speed {dot over (r)}_(t).

Additionally or alternatively, the front-host vehicle distance r_(t) andthe front-host vehicle speed {dot over (r)}_(t) (or front vehicle baseinformation) is determined or otherwise obtained at the traffic signalsystem 20 through the use of one or more sensors of the RSE 26 (or ofthe traffic signal system 20), such as a video detector, other image orelectromagnetic sensor, or inductance loop sensor. Additionally oralternatively, the front-host vehicle distance r_(t) and the front-hostvehicle speed {dot over (r)}_(t) (or front vehicle base information) canbe determined or otherwise obtained by the traffic signal system 20through the traffic signal system 20 receiving one or more wirelesssignals from the front vehicle 11 in embodiments where the front vehicle11 is a connected vehicle (CV). In some embodiments where the frontvehicle 11 is a CV, the front vehicle 11 can use a short-range wirelesscommunications (SRWCs) circuit or chipset that enables SRWCs (e.g.,Wi-Fi™, Bluetooth™). The traffic signal system 20 may then send thefront-host vehicle distance r_(t) and the front-host vehicle speed {dotover (r)}_(t) (or front vehicle base information) to the host vehicle 10or the AV 12 via V2I or other suitable communications. Or, in anotherembodiment, the front vehicle 11 and/or the traffic signal system 20 cansend the front vehicle position (with respect to earth or the trafficlight) (e.g., GNSS position) to the AV 12, which can then use the hostvehicle position (e.g., GNSS position) to determine the front-hostvehicle distance r_(t) and the front-host vehicle speed {dot over(r)}_(t) (or front vehicle base information).

In some embodiments, the host vehicle 10 and/or AV 12 can include one ormore on-board sensors that are capable of obtaining information that iscapable of being used to determine the front-host vehicle distance r_(t)and the front-host vehicle speed {dot over (r)}_(t) (or front vehiclebase information or other front vehicle state information)—for example,the host vehicle 10 can include a digital camera that is positioned toface an area in front of the host vehicle 10 so that images of the frontvehicle may be captured. As another example, the AV 12 can include oneor more sensors (e.g., radar, lidar, and/or camera) that capturesinformation pertaining to the front vehicle 11, such as front vehiclebase information that can be used to calculate or determine thefront-host vehicle distance r_(t) and the front-host vehicle speed {dotover (r)}_(t). Certain techniques can then be used to determine adistance between the host vehicle 10 and the front vehicle 11 based onthe captured images (or image data). As an example, an image recognitiontechnique may be employed that identifies an area of the image datacorresponding to a license plate of the front vehicle. The standard sizeof the license plate may also be known and then used (along with cameraattributes, for example) to determine the distance between the frontvehicle and the host vehicle. Moreover, the change in front-host vehicledistance r_(t) over time can be used to determine the front-host vehiclespeed {dot over (r)}_(t).

The front-host vehicle distance r_(t) and the front-host vehicle speed{dot over (r)}_(t) (or front vehicle base information) can be receivedor otherwise obtained by the one or more remote servers 17. The remoteserver(s) 17 can receive the front-host vehicle distance r_(t) and thefront-host vehicle speed {dot over (r)}_(t) (or front vehicle baseinformation) from the traffic signal system 20 or from the front vehicle11 or the host vehicle 10. It should be appreciated that the front-hostvehicle distance r_(t) and the front-host vehicle speed {dot over(r)}_(t) do not necessarily have to be obtained in the same manner. Themethod 200 then continues to step 240.

It should be appreciated that the various variables discussed above areall reflect their information taken at or very close to the same time(i.e., within a maximum allowable time difference), which can be, forexample, 2 seconds, 1 second, 500 milliseconds, 250 milliseconds, 125milliseconds, 100 milliseconds, or 50 milliseconds. The times reflectedby each of these variables is referred to as an “associated time” and,when the associated times are within a maximum allowable timedifference, it can be said that each of these associated timescorrespond to a time when the human-driven vehicle approaches thetraffic signal. For example, the associated time of the hostvehicle-traffic light distance d_(x) represents the time at which thevehicle is determined to be a distance d_(x) from the traffic light (orstop line). As another example, the associated time of the traffic lightsignal phase P_(t) is a time at which (or is a range of time thatincludes a time at which) the status of the traffic light is determinedto be phase P_(t). As yet another example, the associated time of theelapsed phase time is a time at which (or is a range of time thatincludes a time at which) the amount of time between the current phaseand the previous phase is determined to be equal to T_(t). The maximumallowable time difference of any one of the associated times withrespect to any other one of the associated times can be set based on theparticular application or system in which the method is beingimplemented, as will be appreciated by those skilled in the art. In oneembodiment, the maximum allowable time difference can be predeterminedfor a particular application. In other embodiments, the maximumallowable time difference can be dynamically adjusted based on thetraffic signal or intersection that the vehicle is approaching. In atleast some embodiments, the closer in time that these associated timesare with respect to one another, the more accurate the determination ofthe predicted trajectory.

In step 240, a time of day TOD is determined that represents a time ofthe day of the associated times of the variables obtained in steps210-230. The detail or granularity of the time of day TOD can be setbased on the particular AI vehicle trajectory prediction model beingused. The granularity of the time of day TOD can be by hour (e.g.,TOD=5, when it is 5 AM), by the half-hour (e.g., TOD=5.5, when it is5:30 AM), by the minute (e.g., TOD=5.1, when it is 5:06 AM), or by anyother suitable degree as appreciated by those skilled in the art. Thisgranularity may depend on the particular application in which the timeof day TOD will be used. For example, a country road may not havetraffic precipitate as quickly as a city or downtown road and, thus, thegranularity used in the AI vehicle trajectory prediction model for thecity or downtown road may be much smaller than that used for the countryroad—for example, the granularity in the time of day TOD for the city ordowntown road may be by the minute whereas the granularity in the timeof day TOD for the country road may be by the hour. The time of day TODvalue can be obtained based on the associated times of the variousvariables and may be determined by the device or system carrying out themethod 200, such as the remote server(s) 17 or the AV 10. The method 200continues to step 250.

In step 250, the host vehicle state X_(t) ^(HV), the traffic light stateX_(t) ^(TL), and the front vehicle state X_(t) ^(FV) are provided asinputs into an artificial intelligence (AI) vehicle trajectoryprediction application. This can include, for example, providing thehost vehicle-traffic light distance d_(x), the longitudinal host vehiclespeed v_(x), the traffic light signal phase P_(t), the traffic lightsignal timing T_(t), the front-host vehicle distance r_(t), thefront-host vehicle speed {dot over (r)}_(t), and the time of day TODinto the artificial intelligence (AI) vehicle trajectory predictionapplication. The AI vehicle trajectory prediction application can beembodied by computer instructions stored on memory. These computerinstructions can be executed by a processor of the remote server(s) 17or the host vehicle 10 (or other suitable device) and, when the computerinstructions are executed, the executing device or system (e.g., theremote server(s) 17, the host vehicle 10) then can provide input intothe AI vehicle trajectory prediction application, which can then processthe inputs according to the AI vehicle trajectory prediction model thatthe AI vehicle trajectory prediction application is built on andimplements. The method 200 continues to step 260.

In step 260, the predicted trajectory of the human-driven host vehicleis determined using the AI vehicle trajectory prediction application.The output of the AI vehicle trajectory prediction application can berepresented in a number of ways, and includes information that is usedto determine the predicted trajectory of the host vehicle. In oneembodiment, the output of the AI vehicle trajectory predictionapplication is the predicted trajectory and, in other embodiments, theoutput of the AI vehicle trajectory prediction application isinformation used to obtain or derive the predicted trajectory. Also, itshould be appreciated that the predicted trajectory can be representedin a number of different ways. For example, the output can be anacceleration, such as a longitudinal acceleration a_(x), of thehuman-driven host vehicle, which includes the direction and magnitude ofthe acceleration. The acceleration a_(x) can be used to determine thepredicted trajectory as well as other properties of the human-drivenhost vehicle, such as its velocity (including direction and magnitude),as well as its position over time. In another example, the output of theAI vehicle trajectory prediction application can be a time-position plot(or a collection of time, position pairs) and this can be used torepresent the predicted trajectory and the position of the human-drivenhost vehicle over time. The method 200 then ends.

In some embodiments, any one or more of steps 210-260 can be carried outover a plurality of iterations so as to obtain a more accurate predictedtrajectory. For example, a prediction window can be predetermined (ordetermined at a time prior to or during one or more steps of themethod), and the inputs (those obtained in steps 210-240) can beobtained a plurality of times (e.g., one time for each iteration), suchas according to a predetermined interval. Numerous predictedtrajectories can then be obtained and then used to obtain a finalpredicted trajectory, which can include averaging each of the predictedtrajectories, averaging each of the predicted trajectories according toan associated confidence score (i.e., each prediction is weighted basedon its associated confidence score), or according to other methods.

In some embodiments, the AI vehicle trajectory prediction applicationcan take other variables into consideration, such as the number of lanesof the roadway at the intersection, the type of traffic signal, thespeed limit of the roadway, past human-driving behavior at one or moretraffic signals, etc. In such embodiments, the AI vehicle trajectoryprediction model can be learned or generated based on the inputs to beprovided into the AI vehicle trajectory prediction application.

In some embodiments, the method 200 may include determining whether afront vehicle is present; that is, whether there is a vehicle betweenthe host vehicle and the intersection that the host vehicle isapproaching. For example, the host vehicle 10 may use onboard sensors,such as a radar, lidar, or camera, to determine whether a front vehicleis present. Or, the traffic signal system 20 may use sensors todetermine whether a front vehicle is present, which may communicate thisto the host vehicle 10. When it is determined that a front vehicle isnot present, then certain default or predetermined values may be set asthe front vehicle state X_(t) ^(FV) in place of values that wouldactually be determined were a front vehicle present or detected. Inanother embodiment, the host vehicle 10, the AV 12, or otherdevice/system carrying out the method 200 may select a different AIvehicle trajectory prediction application or policy, such as one thatdoes not take into consideration information concerning a front vehicle.

In at least some embodiments, the predicted trajectory and/or otherinformation derived therefrom can be sent to one or more other devices.For example, in embodiments where the remote server(s) 17 carry outsteps 250 and 260, the predicted trajectory can be sent to the frontvehicle 11, the AV 12, or other nearby vehicle, which can then use thepredicted trajectory for purposes of autonomous driving and/or for otheruses. For example, in one embodiment, the V 12 receives the predictedtrajectory of the host vehicle 10 and then determines an autonomousvehicle action based on the predicted trajectory, such as determining aplanned trajectory for the AV 12 to propel itself along. The autonomousvehicle action can be actuating a braking device of the AV 12, actuatinga steering mechanism of the AV 12, or accelerating the AV 12. In someembodiments, the AV 12 carries out the method 200, which can include theAV 12 receiving the host vehicle state from the host vehicle (e.g., viaV2V communications or through detection using onboard sensors of the AV)and the traffic signal state from the traffic signal system 20 (or fromthe host vehicle 10 or remote server). Also, the AV 12 can receive thefront vehicle state (or front vehicle base information) from the frontvehicle 11 (e.g., via V2V communications or through detection usingonboard sensors of the AV), or from the host vehicle 10.

In some embodiments, a next phase time may be used as a part of theinput into the AI vehicle trajectory prediction application. The nextphase time is the amount of time until the next traffic signal phasechange (e.g., when the light if RED, the next phase time is the amountof time until the light changes to GREEN). In one embodiment, the nextphase time can be used in place of the traffic light signal timingT_(t). And, in another embodiment, a predicted next phase time may beused as a part of the input into the AI vehicle trajectory predictionapplication. The predicted next phase time is the predicted amount oftime until the next traffic signal phase change (e.g., when the light ifRED, the next phase time is the predicted amount of time until the lightchanges to GREEN). This may be useful for scenarios where the trafficsignal phase timing is dynamically determined, such as when the trafficsignal phase timing depends on whether an inductance loop detector hasdetected a vehicle at the traffic signal.

Additionally, in some embodiments, the predicted trajectory can bestored in a database or otherwise at the remote facility 16 (or otherremote facility (e.g., municipal facility 18)) and used for variouspurposes. One such purpose can be to further or continuously train andupdate the AI vehicle trajectory prediction model and application. Insome such embodiments, the actual trajectory of the human-driven vehiclecan be determined at some later time (i.e., a time after the method 200)and then this actual trajectory can be compared to the predictedtrajectory so as to inform the system of the accuracies of theprediction. This actual trajectory can thus be utilized as feedback tothe AI vehicle trajectory prediction model and application, and thenused to further improve the AI vehicle trajectory prediction model andapplication.

Results. In this section, prediction results are presented for all thebase scenarios (i.e., G, Y, R, GY, YR, RG), and an extended scenario GYRto evaluate long-term performances. The resulting trajectories of thedeterministic model (or policy) (or a model or policy that predicts themost-probable trajectories) for the scenarios G, Y, R are presented inFIGS. 8A-D, 9A-D, and 10A-D. The resulting trajectories of deterministicmodel (or policy) for the scenarios GY, YR, RG are depicted in FIGS.11A-D, 12A-D, and 13A-D. FIGS. 8A-D depict trajectories of thedeterministic model for the scenario G according to two samples, withthe first sample corresponding to FIGS. 8A-B and the second samplecorrespond to FIGS. 8C-D. Likewise, FIGS. 9A-B depict trajectories ofthe deterministic model for the scenario Y according to a first sample,FIGS. 9C-D depict trajectories of the deterministic model for thescenario Y according to a second sample, FIGS. 10A-B depict trajectoriesof the deterministic model for the scenario R according to a firstsample, FIGS. 10C-D depict trajectories of the deterministic model forthe scenario R according to a second sample. Also, in a likewise manner,FIGS. 11A-D, 12A-D, and 13A-D depict trajectories of the deterministicmodel for the scenarios GY, YR, and RG, respectively, for two samples.FIGS. 14A-B depict trajectories of the deterministic model for thescenario GYR for one sample. For FIGS. 8A-13D, all trajectories wereobtained at time t=0 by running the iterative predictions andpropagations of the dynamics every 0.2 s, over the prediction horizon (5s).

Each of FIGS. 8A, 8C, 9A, 9C, 10A, 10C, 11A, 11C, 12A, 12C, 13A, 13C,and 14A depict a time-distance plot showing the actual trajectory (solidline) (802 a, 810 a, 902 a, 910 a, 1002 a, 1010 a, 1102 a, 1110 a, 1202a, 1210 a, 1302 a, 1310 a, 1402 a), the predicted trajectory (dashedline) (802 b, 810 b, 902 b, 910 b, 1002 b, 1010 b, 1102 b, 1110 b, 1202b, 1210 b, 1302 b, 1310 b, 1402 b), and the traffic light (804, 812,904, 912, 1004, 1012, 1104, 1112, 1204, 1212, 1304, 1312, 1404). Each ofFIGS. 8B, 8D, 9B, 9D, 10B, 10D, 11B, 11D, 12B, 12D, 13B, 13D, and 14Bdepict a time-speed plot and a time-acceleration plot showing the actualspeed (solid line) (806 a, 814 a, 906 a, 914 a, 1006 a, 1014 a, 1106 a,1114 a, 1206 a, 1214 a, 1306 a, 1314 a, 1406 a), the predicted speed(dashed line) (806 b, 814 b, 906 b, 914 b, 1006 b, 1014 b, 1106 b, 1114b, 1206 b, 1214 b, 1306 b, 1314 b, 1406 b), the actual acceleration(solid line) (808 a, 816 a, 908 a, 916 a, 1008 a, 1016 a, 1108 a, 1116a, 1208 a, 1216 a, 1308 a, 1316 a, 1408 a), the predicted acceleration(dashed line) (808 b, 816 b, 908 b, 916 b, 1008 b, 1016 b, 1108 b, 1116b, 1208 b, 1216 b, 1308 b, 1316 b, 1408 b). In FIGS. 11A, 11C, 12A, 12C,13A, 13C, and 14 the vertical dashed line indicates a phase change ofthe traffic signal.

From the training data, a large number of distinct sample trips for eachbase scenario and the extended scenario were obtained. Among all thetrips, six (6) sample trips for the scenarios GY, YR, and RG wereselected and their prediction results are depicted in FIGS. 11A-D,12A-D, and 13A-D. FIGS. 8A-D, 9A-D, and 10A-D show deterministic (ormost-probable) trajectory predictions on sample trips for the scenarioG, Y, and R. The two scenario G examples depict when vehicle coasts ingreen phases through the signalized intersections. The scenario Y in thefirst example (FIGS. 9A-B) depicts an instance when vehicles slows downas it approaches to the intersection. The scenario Y in the secondexample (FIGS. 9C-D) describes an instance where a vehicle passesthrough the intersection, maintaining its speed. The scenario R in thefirst example (FIGS. 10A-B) describes an example when a vehicle is atstop. The scenario R in the second example (FIGS. 10C-D) describes aprediction instance where a vehicle slows down as it approaches to theintersection.

FIGS. 11A-D, 12A-D, and 13A-D shows deterministic (or most-probable)trajectory predictions on sample trips for the scenarios GY, YR, and RG.The scenario GY in the first example (FIGS. 11A-B) presents an instancewhen a vehicle reacts to a phase shift to yellow and decides to stopbefore the intersection. The scenario GY in the second example (FIGS.11C-D) depicts an instance when a vehicle decides to pass through theintersection by speeding up. The scenario YR in the first example (FIGS.12A-B) depicts an instance when a vehicle slows down as it approaches tothe intersection. The scenario YR in the first example (FIGS. 12C-D) isan example where the human driver chose to pass through the intersectionin a RED phase. The prediction algorithm was able to predict thisbehavior that violates a traffic rule. The scenario RG in the firstexample (FIGS. 13A-B) describes an example when a vehicle departs as thephase shifted to GREEN. Here, one could infer that the vehicle is at aqueue based on the position of the vehicle and the length of the vehicleas it is stationary. Again, the prediction algorithm described hereinwas able to predict the moment when the vehicle started the departure,capturing the existence of a queue formed near the intersection. Thescenario RG in the first example (FIGS. 13C-D) describes an instancewhere the phase was originally red and shifted to green, which made thevehicle slowed down for the first few seconds.

FIGS. 14A-B shows a deterministic (or most-probable) trajectoryprediction on a sample trip for the GYR scenario. The prediction horizonis 15 s, meaning that the 15 s long trajectory were obtained at t=0.Predictions were made every 0.2 s. Although the prediction horizon (15s) is much longer than the sample trips in FIGS. 8A-13D, themost-probable position, speed, and acceleration predictions werequalitatively almost identical to the actual position, speed, andacceleration profile of the human driver. Note that transient responseof the human-driver in the phase shift from green to yellow (as shown asthe delayed deceleration, around t=3.5 s), and general trends in thephase shifts GY and YR were captured.

Results for particular models. The discussion below is divided into four(4) sections. In Section A, the resulting trajectory predicts arepresented for 4 examples sampled from our dataset to visualize theimpact of X^(TL). Furthermore, the utilization of phases and timings ofTLs is shown to help the proposed model to accurately predict long-termtrajectories. In Section B, a set of metrics to evaluate the performanceof trajectory predictions are described. And, an ablation study isdiscussed Section C. The ablation study was designed to test the 4variants of our deterministic models (ƒ_(d), ƒ_(d) ^(NoFV), ƒ_(d)^(NoTL), ƒ_(d) ^(NoFVTL)) on the test set (total 3,111 sample episodesof the seven scenarios discussed above). In Section D, a demonstrationis provided on how the probabilistic prediction algorithm can beutilized to tackle a scenario with competing policies, such as theyellow-light dilemma zone scenario, which is discussed below.

Section A. In order to investigate the impact of X^(TL), four (4)variants of the deterministic policy models are trained, and those four(4) models are each used to compute the trajectory predictions over afixed prediction horizon via Equation 6. The four policy models areƒ_(d), ƒ_(d) ^(NoFV) ƒ_(d) ^(NoTL) ƒ_(d) ^(NoFVTL), which are each nameddepending on which features are used as the context C. Denoting C^(mode)as the context input for a deterministic policy model ƒ_(d) ^(mode),C^(NoFVTL):=[TOD], C^(NoTL):=[X^(FV), TOD], C^(NoFV):=[X^(TL), TOD],C:=[X^(FV), X^(TL), TOD]. The proposed models of ƒ_(d), and ƒ_(d)^(NoFV) provide more accurate predictions of the trajectory of the hostvehicle, which is discussed more below.

FIGS. 15A-18B presents the trajectory predictions for four (4) examples(1 RG (FIGS. 15A-15B), 1 YR (FIGS. 16A-16B), 2 GYR scenarios (FIGS.17A-18B)) sampled from our dataset. The four trajectories that aredepicted in each scenario represents ground-truth, three X_(0:t) ^(HV)each from ƒ_(d), ƒ_(d) ^(NoFV), and ƒ_(d) ^(NoTL). ƒ_(d) ^(NoFV) doesnot use the front vehicle state X_(t) ^(FV) as input, and ƒ_(d) ^(NoTL)does not use the traffic signal state X_(t) ^(TL) as input. The sampleshown in FIGS. 15A-15B is similar to the motivational example in FIGS.2A-2B. At t=0, the driver was stopped at a traffic light or signal inthe red phase. It is reasonable to assume that a model without X_(0:5s)^(TL) is likely to predict the vehicle to stay put or not move. Thegraph shown in FIG. 15A depicts predictions of the vehicle's positionover time and the graph shown in FIG. 15B depicts predictions of thevehicle's longitudinal speed over time during a red-green (RG) scenarioin which the traffic signal is in the red phase at the beginning of theprediction window and then changes to green (at time t=1.5 s in thescenario shown) as indicated by the bar 1504. This bar 1504 alsorepresents the position of the traffic signal from the vehicle (about9.5 m at time t=0 s in the scenario shown). The solid lines 1502 a, 1506a show the actual position and speed, respectively, over time of thevehicle. The dashed lines 1502 b, 1506 b show the prediction of thevehicle position and speed, respectively, over time when using thecontext vector C having the inputs C:=[X^(FV) X^(TL), TOD]. The dashedline 1502 c, 1506 c shows the prediction of the vehicle position andspeed, respectively, over time when using the context vector C^(NoFV)and the dashed line 1502 d, 1506 d shows the prediction of the vehicleposition and speed, respectively, over time when using the contextvector C^(NoTL). As expected, the prediction from ƒ_(d) ^(NoTL)(indicated at 1502 d, 1506 d) failed to predict X^(HV) accurately,whereas the other two models were able to produce predictions close tothe ground-truth.

The plots of FIGS. 16A-16B are for a yellow-red (YR) scenario where thedriver slowed down approaching the intersection. The graph shown in FIG.16A depicts predictions of the vehicle's position over time and thegraph shown in FIG. 16B depicts predictions of the vehicle'slongitudinal speed over time during a YR scenario in which the trafficsignal is in the yellow phase at the beginning of the prediction windowand then changes to red (at about time t=2.5 s in the scenario shown) asindicated by the bar 1604. This bar 1604 also represents the position ofthe traffic signal from the vehicle (about 88 m at time t=0 s in thescenario shown). The solid lines 1602 a, 1606 a show the actual positionand speed, respectively, over time of the vehicle. The dashed lines 1602b, 1606 b show the prediction of the vehicle position and speed,respectively, over time when using the context vector C having theinputs C:=[X^(FV) X^(TL), TOD]. The dashed line 1602 c, 1606 c shows theprediction of the vehicle position and speed, respectively, over timewhen using the context vector C^(NoFV), and the dashed line 1602 d, 1606d shows the prediction of the vehicle position and speed, respectively,over time when using the context vector C^(NoTL). Given P₀=Y (theinitial phase is yellow) and that the vehicle was cruising (v₀=15),ƒ_(d) ^(NoTL) predicts that the vehicle will maintain this speed,causing the prediction errors to grow over time. The other two modelsƒ_(d), ƒ_(d) ^(NoFV) which use X_(0:5s) ^(TL) took account for the phaseshift at t=2.4 s, and accurately predict how the driver would react tothe shift.

On the other hand, FIGS. 17A-18B describe scenarios that are long (about15 s), and span a full cycle of phases or a green-yellow-red (GYR)scenario. The first example shown in FIGS. 17A-17B describes a scenariowhere the vehicle was initially at stop due to a queue formed at theentrance of the intersection. The graph shown in FIG. 17A depictspredictions of the vehicle's position over time and the graph shown inFIG. 17B depicts predictions of the vehicle's longitudinal speed overtime during a GYR scenario in which the traffic signal is in the greenphase at the beginning of the prediction window, then changes to yellow(at about time t=0.5 s in the scenario shown), and then changes to red(at about time t=4.9 s in the scenario shown), as indicated by the bar1704. This bar 1704 also represents the position of the traffic signalfrom the vehicle (about 135 m at time t=0 s in the scenario shown). Thesolid lines 1702 a, 1706 a show the actual position and speed,respectively, over time of the vehicle. The dashed lines 1702 b, 1706 bshow the prediction of the vehicle position and speed, respectively,over time when using the context vector C having the inputs C:=[X^(FV)X^(TL), TOD]. The dashed line 1702 c, 1706 c shows the prediction of thevehicle position and speed, respectively, over time when using thecontext vector C^(NoFV), and the dashed line 1702 d, 1706 d shows theprediction of the vehicle position and speed, respectively, over timewhen using the context vector C^(NoTL). Given the green phase observedat t=0, ƒ_(d) ^(NoTL) (indicated at 1702 d, 1706 d) predicted that thequeue will be dissipated soon and the vehicle will start to move in thefuture. However, what really happened was that the phase shifted to redshortly, made the vehicle stay put (ground-truth). Note, the two modelswhich utilized X_(0:15s) ^(TL) were able to accurately predict thetrajectory over the 15 second prediction horizon.

The second example shown in FIGS. 18A-18B is for another GYR scenario,but with the vehicle initially approaching the intersection at v₀=16(m/s). The model ƒ_(d) ^(NoTL) predicted that the vehicle will cross theintersection, given P_(t=0)=G (the initial phase is green). In reality,the vehicle made a stop before the intersection. Again, ƒ_(d), ƒ_(d)^(NoFV) accommodated the future phases and timings of TL, successfullypredicted that the vehicle would make a stop before the intersection.The graph shown in FIG. 18A depicts predictions of the vehicle'sposition over time and the graph shown in FIG. 18B depicts predictionsof the vehicle's longitudinal speed over time during a GYR scenario inwhich the traffic signal is in the green phase at the beginning of theprediction window, then changes to yellow (at about time t=0.5 s in thescenario shown), and then changes to red (at about time t=4.9 s in thescenario shown), as indicated by the bar 1804. This bar 1804 alsorepresents the position of the traffic signal from the vehicle (about135 m at time t=0 s in the scenario shown). The solid lines 1802 a, 1806a show the actual position and speed, respectively, over time of thevehicle. The dashed lines 1802 b, 1806 b show the prediction of thevehicle position and speed, respectively, over time when using thecontext vector C having the inputs C:=[X^(FV) X^(TL), TOD]. The dashedline 1802 c, 1806 c shows the prediction of the vehicle position andspeed, respectively, over time when using the context vector C^(NoFV),and the dashed line 1802 d, 1806 d shows the prediction of the vehicleposition and speed, respectively, over time when using the contextvector C^(NoTL). As demonstrated, the impact of X^(TL) is significant:uncertainties in X^(TL) can cause high prediction errors, especially inlong-term predictions. The results suggest that the existing predictingmethods can perform poor without the knowledge of future X^(TL),highlighting why the problem is critical and needs to be solved. Theproposed models can be used to solve this problem; that is, the modelswhich utilize future X^(TL) are able to predict reactions of humandrivers to traffic signals or lights (TLs) in diverse scenarios andgreatly improve the accuracy of the predictions.

Section B. In order to make fair performance comparisons, the followingthree evaluation metrics were used: mean absolute error (MAE), timeweighted absolute error (TWAE), and absolute deviation at the end of theprediction window (ADN) defined in Equations 8, 9, and 10, where{circumflex over (X)}_(k) ^(HV), X_(k) ^(HV) each represents thepredicted and the actual state at the k-th step. For the graphicaldescription, refer to FIG. 19 .

$\begin{matrix}{{MAE}:=\frac{\Sigma_{k = 1}^{N}{{{\overset{\hat{}}{X}}_{k}^{HV} - X_{k}^{HV}}}}{N}} & ( {{Equation}\mspace{14mu} 8} ) \\{{TWAE}:=\frac{\Sigma_{k = 1}^{N}( {t_{k}{{{\overset{\hat{}}{X}}_{k}^{HV} - X_{k}^{HV}}}} )}{\Sigma_{k = 1}^{N}t_{k}}} & ( {{Equation}\mspace{14mu} 9} ) \\{{{AD}N}:={{{\overset{\hat{}}{X}}_{N}^{HV} - X_{N}^{HV}}}} & ( {{Equation}\mspace{14mu} 10} )\end{matrix}$∀k: Δt_(k)=0.2 s, τ=2 s was used (history). For a scenario with aprediction window T=5 s (or t_(N)=5 s), the last index N is 25.

Section C. The goal of the ablation study is to investigate the impactof X^(TL) in the trajectory prediction problem and evaluate the proposedmodels quantitatively. The resulting box plots for the statistics on theperformance of the deterministic (or most-probable) trajectorypredictions for 6 scenarios (G, R, GY, YR, RG, GYR) are shown in FIGS.20-25 .

In FIGS. 20-25 , there is shown evaluations on ADN for the four models(ƒ_(d), ƒ_(d) ^(NoFV), ƒ_(d) ^(NoTL), ƒ_(d) ^(NoFVTL), which correspondto “All”, “NoFV”, “NoTL”, “NoFVTL”, respectively) on the test set. Thesample size for the scenarios are 688 for scenario G (FIG. 20 ), 1909for scenario R (FIG. 21 ), 68 for scenario GY (FIG. 22 ), 81 forscenario YR (FIG. 23 ), 362 for scenario RG (FIG. 24 ), and 32 forscenario GYR (FIG. 25 ), totaling 3,111 sample episodes. Note, thescenario Y is not depicted due to the inconsistent and short predictionhorizon. During testing, it was observed that the phase Y usually lastsanywhere between 2.5 s to 4 s. The first 2 s are used as inputs, whichmeans the prediction horizon for the scenario Y is only 0.5 s to 2 s.Second, we present the results on the performance evaluation of the fourmodels on MAE, TWAE, and ADN on the same test set. Tables I and II(FIGS. 26-27 ) each serves the result for position and velocity errors.

ADN has the largest error among the three metrics (the magnitude oferror: MAE<TWAE<ADN). FIGS. 20-25 each depicts a box plot for ADN. In abox plot, the top and bottom edges of the box represents the 1^(st),3^(rd) quartiles, and the median is represented by the horizontal linein the middle of the band. The ends of the whisker extend to data thatis outside typical values but that are not considered outliers (referredto as “extreme points”). The outliers are indicated by ‘+’.

As shown in FIGS. 20-25 , across all scenarios, the two models ƒ_(d),ƒ_(d) ^(NoFV) which utilizes X^(TL) outperform the other two modelsƒ_(d) ^(NoTL), ƒ_(d) ^(NoFVTL), which don't take advantage of the futurephases and timings information. Interestingly, the winner in theexemplary scenarios is not ƒ_(d), but it is ƒ_(d) ^(NoFV), whichperforms the best on all characteristics of boxplot including the1^(st), 3^(rd) quartiles, the median, and the upper limit of the extremepoints. Our interpretation is that the exclusion of X^(FV) increases theprediction accuracy, due to the uncertainty in X_(t>0) ^(FV).

The numbers presented in Tables II and III (FIGS. 26-27 ) agree with (orcorroborate) the results from FIGS. 20-25 across all scenarios. At leastin the exemplary scenario, ƒ_(d) ^(NoFV) is the winner for almost allmetrics, or at least on par with ƒ_(d). In summary, the knowledge offuture states of traffic lights significantly increase the accuracy oftrajectory predictions, as evidenced in the ablation studies: trajectorypredictions with the winner model have roughly 2-6 times smaller(position) MAE, TWAE, AND for T=5 s scenarios (G, R, GY, YR, RG), androughly 9-21 times smaller MAE, TWAE, AND for T=15 s scenario (GYR),compared to trajectory predictions via ƒ_(d) ^(NoTL).

Section D. The outliers observed in FIGS. 20-25 occur mostly because ofedge cases and competing policies. Examples of the edge cases include adriver approaching the intersection in P_(∀t)=R with a high speed andexecuting a sudden break right before the entrance of the intersectionrather than gradually slowing down it approaches the intersection.Another example is that a driver in the middle of the road in P_(∀t)=Gmoving much slower than the average speed of the traffic for unknownreasons. The outliers occur from competing policies are exemplified bythe yellow light dilemma scenario where a driver can either cross theintersection or stop before the intersection.

FIGS. 28-29 describe a sample trip observed in our dataset thatrepresents the yellow light dilemma scenario in which the traffic signal(indicated at 2602 and 2702 in FIGS. 28 and 29 , respectively) changesfrom yellow to red (at about time t=3.5 s). In FIG. 28 , the trafficsignal is at position 80 m as indicated at 2604. As shown in FIG. 28 ,the most-probable trajectory prediction obtained using ƒ_(d) ^(NoFV)(indicated at 2602 b) predicts that the vehicle would make a stop beforethe intersection; however, the driver actually crossed the intersectioneven after the phase shifted to red (as indicated at 2602 a). This is anexample where the proposed probabilistic human policy models come inhandy. As shown in FIG. 29 , the prediction made by a probabilisticmodel includes a first MDN peak (indicated at 2702 b) in which thevehicle is predicted to stop at the traffic signal and a second MDN peak(indicated at 2702 c) in which the vehicle is predicted to proceedthrough the traffic signal. The actual position is represented at 2702a. The traffic signal is at position 80 m as indicated at 2704.

The probabilistic models are capable of reproducing a multi-modaldistributions, thus capture the other competing policy (cross). Theprobabilistic predictions are not only capable of capturing competingpolicies, but also able to reason the uncertainties of the predictionsand provide contexts on the predictions as the probabilities of thepredictions can be estimated. However, in at least some scenarios, thedeterministic models are still useful: the solutions (at least accordingto some embodiments) are simple, cost-efficient, and easy to interpret.They can serve as nominal trajectories of human drivers in situationsthat can be approximated uni-modal or described with a normaldistribution. The nominal trajectories can be used in a trajectoryplanning algorithm which does not allow uncertainties in surroundingenvironments. For the scenarios with 5 s prediction horizon, the time tocompute a most-probable trajectory prediction is less than 10 msecs on asingle-core personal laptop with i7-6500U 2.50 GHz CPU, and 8 GB RAMwithout a parallelization. However, it takes several seconds (5-10 s for1,000 rollout trajectories) to construct the pdf for the probabilisticpredictions on the same machine due. One can significantly reduce thetime via parallel computing (GPU).

One of the remaining challenges for autonomous driving is how toaccurately predict trajectories of vehicles near traffic lights invarious states of traffic lights. The trajectory prediction problemwhere the existing methods perform poor was described above. Thisproblem addressed this gap in conventional systems by proposing anapproach where the future states of traffic lights are leveraged.Specifically, both a deterministic and probabilistic human policy modelwere proposed, each of which simulate how human drivers take actions interms of accelerations depending on diverse driving scenarios near TLs.These models are trained on a real-world naturalistic driving datasetand are utilized to obtain longitudinal trajectory predictions based onthe idea of utilizing the future traffic signal phase and timinginformation obtained from V2I communications. The human policy modelsare learned using a recurrent neural network, and a Gaussian-mixturebased Mixture Density Network. It is shown that the utilization offuture phases and timings of TLs can significantly improve the qualityof the trajectory predictions through the ablation study. The ablationstudy, which was discussed above, also highlights that the proposedmethods are comprehensive models that tackle the diverse scenario of thetrajectory prediction problem described above. It is worth noting that,at least in some embodiments or implementations, the proposed models arenot necessarily replacements for other state of the art trajectoryprediction models, but rather the great addition to any method thatconcerns vehicle trajectory predictions near TLs.

The predicted trajectories then can be utilized for various applicationsin decision makings, trajectory planning, and controls of a host vehicle(either a self-driving car or a human-driven car) or another vehicle.The above models can be used as an extension of the work presented in G.Oh and H. Peng, “Eco-driving at signalized intersections: What ispossible in the real-world?” in 2018 21st International Conference onIntelligent Transportation Systems (ITSC). IEEE, 2018, pp. 3674-3679;the energy-optimal planning algorithm presented in that paper can beimproved by leveraging the trajectory prediction framework (discussedabove) to model other vehicles in the scenes. In conclusion, theproposed human policy model helps one better understand and predictbehaviors of human drivers in the vicinity of traffic lights, and can beleveraged to improve autonomous driving in urban city driving, includingdecision-making, planning, and control of host vehicles.

It is to be understood that the foregoing description is of one or moreembodiments of the invention. The invention is not limited to theparticular embodiment(s) disclosed herein, but rather is defined solelyby the claims below. Furthermore, the statements contained in theforegoing description relate to the disclosed embodiment(s) and are notto be construed as limitations on the scope of the invention or on thedefinition of terms used in the claims, except where a term or phrase isexpressly defined above. Various other embodiments and various changesand modifications to the disclosed embodiment(s) will become apparent tothose skilled in the art.

As used in this specification and claims, the terms “e.g.,” “forexample,” “for instance,” “such as,” and “like,” and the verbs“comprising,” “having,” “including,” and their other verb forms, whenused in conjunction with a listing of one or more components or otheritems, are each to be construed as open-ended, meaning that the listingis not to be considered as excluding other, additional components oritems. Other terms are to be construed using their broadest reasonablemeaning unless they are used in a context that requires a differentinterpretation. In addition, the term “and/or” is to be construed as aninclusive OR. Therefore, for example, the phrase “A, B, and/or C” is tobe interpreted as covering all of the following: “A”; “B”; “C”; “A andB”; “A and C”; “B and C”; and “A, B, and C.”

The invention claimed is:
 1. A method for determining a predictedtrajectory of a human-driven host vehicle as the human-driven hostvehicle approaches a traffic signal, wherein the method is carried outby one or more electronic controllers, and wherein the method comprisesthe steps of: obtaining a host vehicle-traffic light distance d_(x) anda longitudinal host vehicle speed v_(x) that are each taken when thehuman-driven host vehicle approaches the traffic signal; obtaining atraffic light signal phase P_(t) and a traffic light signal timingT_(t), wherein the traffic light signal phase P_(t) represents a phaseof the traffic signal taken when the human-driven host vehicleapproaches the traffic signal, and wherein the traffic light signaltiming T_(t) represents an amount of time elapsed since a last phasechange of the traffic signal taken when the human-driven host vehicleapproaches the traffic signal; obtaining a time of day TOD; providingthe host vehicle-traffic light distance d_(x), the longitudinal hostvehicle speed v_(x), the traffic light signal phase P_(t), the trafficlight signal timing T_(t), and the time of day TOD as input into anartificial intelligence (AI) vehicle trajectory prediction application,wherein the AI vehicle trajectory prediction application implements anAI vehicle trajectory prediction model; and determining the predictedtrajectory of the human-driven host vehicle using the AI vehicletrajectory prediction application.
 2. The method of claim 1, wherein themethod further includes obtaining a front vehicle state X^(FV), whereinthe front vehicle state includes a front-host vehicle distance r_(t) anda front-host vehicle speed {dot over (r)}_(t), and wherein the providingstep further includes providing the front vehicle state X^(FV) as inputinto the AI vehicle trajectory prediction application.
 3. The method ofclaim 2, wherein the front vehicle state X^(FV) is obtained at the oneor more electronic controllers based on front vehicle base informationthat is obtained at the front vehicle and then sent viavehicle-to-vehicle (V2V) communications to the one or more electroniccontrollers.
 4. The method of claim 1, wherein the traffic light signalphase P_(t) and the traffic light signal timing T_(t) are obtained froma traffic light control system that is present at an intersection wherethe traffic light is located.
 5. The method of claim 1, wherein thepredicted trajectory is obtained at an autonomous vehicle that isapproaching the traffic light and that is separate from the human-drivenhost vehicle.
 6. The method of claim 5, wherein the method is carriedout at the autonomous vehicle as the human-driven host vehicleapproaches the traffic light.
 7. The method of claim 6, wherein theautonomous vehicle obtains the traffic light signal phase P_(t) and thetraffic light signal timing T_(t) from a traffic signal system locatedat an intersection where the traffic light is located.
 8. The method ofclaim 7, wherein the autonomous vehicle receives the traffic lightsignal phase P_(t) and the traffic light signal timing T_(t) viavehicle-to-infrastructure (V2I) communications from roadside equipmentthat is a part of the traffic signal system.
 9. The method of claim 6,wherein the autonomous vehicle receives the traffic light signal phaseP_(t) and the traffic light signal timing T_(t) from a traffic signalingcontrol system that is located remotely from the traffic light.
 10. Themethod of claim 6, wherein the host vehicle-traffic light distanced_(x), the longitudinal host vehicle speed v_(x) are obtained at theautonomous vehicle via V2V communications with the host vehicle.
 11. Themethod of claim 1, wherein the vehicle-traffic light distance d_(x), thelongitudinal host vehicle speed v_(x), the traffic light signal phaseP_(t), and the traffic light signal timing T_(t) are each associatedwith an associated time that is no more than a predetermined amountdifferent than another one of the associated times.
 12. The method ofclaim 1, wherein the AI vehicle trajectory prediction model is orincludes a neural network.
 13. The method of claim 12, wherein the AIvehicle trajectory prediction model is a deterministic or a model thatpredicts one or more most-probable trajectories.
 14. The method of claim12, wherein the AI vehicle trajectory prediction model is aprobabilistic model that returns a probability distribution of predictedtrajectories, and wherein the predicted trajectory is obtained bysampling a trajectory from the probability distribution of predictedtrajectories.
 15. The method of claim 14, wherein the neural network isa mixture density network.
 16. The method of claim 12, wherein theneural network is a deep neural network.
 17. The method of claim 1,wherein the method further includes the step of causing an autonomousvehicle to obtain the predicted trajectory of the human-driven vehicle,wherein the autonomous vehicle is configured to: obtain the predictedtrajectory of the human-driven vehicle, and carry out an autonomousvehicle operation based on the predicted trajectory of the human-drivenvehicle.
 18. A method for determining a predicted trajectory of ahuman-driven host vehicle as the human-driven host vehicle approaches atraffic signal, wherein the method is carried out by one or moreelectronic controllers, and wherein the method comprises the steps of:obtaining a host vehicle-traffic light distance d_(x) and a longitudinalhost vehicle speed v_(x) that are each taken when the human-driven hostvehicle approaches the traffic signal, wherein the host vehicle-trafficlight distance d_(x) and the longitudinal host vehicle speed v_(x) eachhave an associated time; obtaining a front vehicle state X^(FV), whereinthe front vehicle state includes a front-host vehicle distance r_(t) anda front-host vehicle speed {dot over (r)}_(t); receiving one or morewireless signals that indicate a traffic light signal phase P_(t) and atraffic light signal timing T_(t), wherein the traffic light signalphase P_(t) represents a phase of the traffic signal taken when thehuman-driven host vehicle approaches the traffic signal, and wherein thetraffic light signal timing T_(t) represents an amount of time elapsedsince a last phase change of the traffic signal taken when thehuman-driven host vehicle approaches the traffic signal, wherein thetraffic light signal phase P_(t) and the traffic light signal timingT_(t) each have an associated time, wherein the associated times of thehost vehicle-traffic light distance d_(x), the longitudinal host vehiclespeed v_(x), the traffic light signal phase P_(t), the traffic lightsignal timing T_(t), the front vehicle state includes a front-hostvehicle distance r_(t), and the front-host vehicle speed {dot over(r)}_(t) are within a maximum allowable time difference with respect toone another; obtaining a time of day TOD; providing the hostvehicle-traffic light distance d_(x), the longitudinal host vehiclespeed v_(x), the traffic light signal phase P_(t), the traffic lightsignal timing T_(t), the front vehicle state X^(FV), and the time of dayTOD as input into an artificial intelligence (AI) vehicle trajectoryprediction application, wherein the AI vehicle trajectory predictionapplication implements an AI vehicle trajectory prediction model, andwherein the AI vehicle trajectory prediction model is or includes aneural network; and determining the predicted trajectory of thehuman-driven host vehicle using the AI vehicle trajectory predictionapplication.
 19. The method of claim 18, wherein the hostvehicle-traffic light distance d_(x) and the longitudinal host vehiclespeed v_(x) are both obtained at an autonomous vehicle through receivingone or more wireless signals from the human-driven host vehicle viavehicle-to-vehicle (V2V) communications.
 20. The method of claim 18,wherein the host vehicle-traffic light distance d_(x) and thelongitudinal host vehicle speed v_(x) are both obtained at theautonomous vehicle through receiving one or more wireless signals from aremote server.